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Executive summary 


What is TIMSS? 


The Trends in International Mathematics and Science Study (TIMSS) is an international 
comparison study of mathematics and science performance, organised by the 
International Association for the Evaluation of Educational Achievement (IEA). The 
study’s main purpose is to provide participating countries with internationally comparable 
data on the performance and attitudes of 9-10-year olds (year 5) and 13-14-year olds 
(year 9) in mathematics and science, together with comparisons of the curriculum and 
the teaching of these subjects in primary and secondary schools. Sixty-four countries and 
benchmarking systems participated in TIMSS 2019. England has participated in every 
TIMSS since the study was first carried out in 1995 and in each subsequent 4-yearly 
cycle’, and the results provide valuable information on trends in England’s absolute and 
relative performance over 24 years. 


In England, testing was conducted with pupils in years 5 and 9 between February and 
June 2019, with a sample of 9,595 pupils across 368 schools. England participated in the 
new eTIMSS format in which the majority of pupils took the tests on tablets, 
complemented by a paper-based bridging study? where a smaller sample of pupils were 
recruited to sit a paper-based test that was used to link the eTIMSS assessment to the 
historic TIMSS assessment scale. Just over half of the participating countries took part in 
eTIMSS. 


England’s year 5 cohort started school in 2013 and had their key stage 1 assessments in 
the summer of 2016°. The year 9 cohort started primary school in 2009 and secondary 
school in 2016, and will take their GCSEs in summer 2021. This Trends in International 
Mathematics and Science Study (TIMSS) 2019: National Report for England focuses on 
comparisons of England’s pupils’ performance and their experiences of mathematics and 
science teaching compared to the highest-performing countries, other English-speaking 
countries and a selection of other European countries. The T/MSS International Report 
2019 offers comparisons across all participating countries‘. 


’ Only year 9 pupils participated in TIMSS assessments in 1999 internationally. 

26,761 pupils in 275 schools participated in eTIMSS in England in 2019; 2,834 pupils in 93 schools 
participated in the bridging study. 

3 See https://www.gov.uk/quidance/2016-key-stage-1-assessment-and-reporting-arrangements- 
ara/section-8-teacher-assessment 

4 See hitp://timssandpirls.bc.edu/timss2019/international-results/ 
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How does the mathematics and science performance of pupils 
in England compare internationally? 


In 2019, pupils in England performed, on average, significantly’ above the TIMSS 
centrepoint (500) in mathematics and science in both years 5 and 9. Compared to 2015, 
England’s performance significantly improved in mathematics at year 5, decreased 
significantly in science at year 9, and remained stable in mathematics at year 9 and 
science at year 5. 


Between 1995 and 2019, the mathematics performance of year 5 and year 9 pupils in 
England has improved. The performance of year 5 pupils in science has been more 
varied, but has also seen some significant improvement over the 24-year period. In year 
9, the science performance of pupils in England has remained broadly static over the 
same period until the decrease in 2019. 


England’s performance in 2019 placed it behind the highest-performing group of 
countries, but significantly above the TIMSS centrepoint in mathematics and science in 
both years 5 and 9. Overall, as in previous years, 5 East Asian countries that participated 
in TIMSS (Chinese Taipei, Hong Kong®, Japan, the Republic of Korea and Singapore), 
together with Russia, performed strongly across both subjects and year groups, although 
there were other countries that performed highly in 1 or more areas. 


5 Where the term ‘significant’ is used this refers to statistical significance. 
® Hong Kong’s pupils’ scores in year 9 science decreased significantly in 2019. 
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Mathematics — year 5 


The trend in England’s year 5 mathematics score is one of improvement over time, from 
significantly below the TIMSS centrepoint in 1995 to significantly above it in 2019’. Figure 
1 shows that the performance of pupils in England has increased in each consecutive 
TIMSS cycle. The increase in England’s average score (10 scale points) between 2015 
and 2019 means year 5 pupils’ performance in 2019 was significantly above that scored 
in all previous TIMSS cycles. 


Figure 1: Trend in average year 5 mathematics score (England) 


Average 
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Source: TIMSS 2019 


Note 1: The 1999 cycle of TIMSS included only year 9 pupils, represented by the dashed 
line. 

Note 2: The 1995 score is an average across the performance of year 4 and year 5 
pupils as the 1995 cycle assessed pupils across both year groups. 

Note 3: Response rates for TIMSS in England were relatively low in 1995 and 2003. 
Note 4: Mathematics scores that represent a significant increase on the previous TIMSS 
cycle are marked with an asterisk (*). 


Seven countries scored significantly higher than England in 2019, 1 country scored at 
broadly the same level and 49 countries scored significantly lower. The same 7 countries 
also performed significantly above England in 2015: the 5 East Asian countries (Chinese 
Taipei, Hong Kong, Japan, the Republic of Korea and Singapore), Northern Ireland and 
Russia. 


7The 1995 score was an average across the performance of year 4 and year 5 pupils. 
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Figure 2 shows that between 1995 and 2019 there has been a significant improvement in 
the proportion of year 5 pupils in England reaching the advanced international 
benchmark and the high and intermediate benchmarks or above®. The proportion of year 
5 pupils reaching the low international benchmark or above remained the same in 2019 
as 2015, having improved significantly between 2011 and 2015. 


Figure 2: Trend in the percentage of year 5 pupils reaching each of the TIMSS 
international benchmarks in mathematics (England) 
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Source: TIMSS 2019. 


Note 1: The 1999 cycle of TIMSS included only year 9 pupils. 

Note 2: The 1995 score is an average across the performance of year 4 and year 5 
pupils as the 1995 cycle assessed pupils across both year groups. 

Note 3: Response rates for TIMSS in England were relatively low in 1995 and 2003. 


In 2019 England had a relatively large difference between its highest- and lowest- 
performing year 5 pupils (a range of 282 scale points). Most of the highest-performing 
and European comparator countries had smaller ranges. 


8 The 1995 score was an average across the performance of year 4 and year 5 pupils. 
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Mathematics — year 9 


The performance of year 9 pupils in mathematics has seen significant improvement over 
the last 24 years, most notably between 2003 and 2007, and has been broadly stable 
since 2007 (see Figure 3)°. The 2019 TIMSS average score for England was 515. 
England’s year 9 pupils continue to score significantly above the TIMSS centrepoint. 


Figure 3: Trend in the average year 9 mathematics score (England) 


Average 
mathematics 
score 


580 
560 


540 


* 518 
515 
520 513 507 


500 


496 498 TIMSS SCALE CENTREPOINT 
480 498 


460 
1995 1999 2003 2007 2011 2015 2019 


Source: TIMSS 2019. 


Note 1: The 1995 score is an average across the performance of year 8 and year 9 
pupils as the 1995 cycle assessed pupils across both year groups. 

Note 2: Response rates for TIMSS in England were relatively low in 1995, 1999 and 
2003. 

Note 3: Scores that represent a significant increase on the previous TIMSS cycle are 
marked with an asterisk. 


Six countries scored significantly higher than England (as in 2015), 7 countries scored at 
a similar level, and 25 countries scored significantly below. There has been no change in 
the 6 countries that performed significantly above England since 2015 — the 5 East Asian 
countries and Russia. 


As in year 5, between 1995 and 2019 there was an improvement in the percentage of 
year 9 pupils in England reaching all the international benchmarks, except the low 


°The 1995 score was an average across the performance of year 8 and year 9 pupils. 
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benchmark or above, a proportion which has remained relatively stable’®. The 
performance of pupils in England was relatively stable across all the benchmarks 
between 2015 and 2019. 


Figure 4: Trend in the percentage of year 9 pupils reaching each of the TIMSS 
international benchmarks in mathematics (England) 
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Source: TIMSS 2019. 


In 2019, England had a relatively large difference between its highest- and lowest- 
performing year 9 pupils (a range of 297 scale points). While still larger than the range of 
scores in all European comparator countries, England’s range was smaller than in some 
of the highest-performing countries: Chinese Taipei, Hong Kong and the Republic of 
Korea. 


TIMSS allows a comparison of a cohort’s performance over 2 cycles, since year 9 pupils 
in 2019 were in year 5 in 2015"'. Relative to the TIMSS centrepoint, this cohort of pupils 
performed better in year 5 than in year 9. This was also the case in 2015. Similar trends 
were also reported in some of the comparator countries, with the exception of the 6 


10 The 1995 score was an average across the performance of year 8 and year 9 pupils. 
11 Although the 2015 year 5 pupils and 2019 year 9 pupils were from the same cohort, different pupils from 
the cohort were selected to take part in TIMSS in each cycle. 
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highest-performing countries, which generally either maintained their positions or 
achieved greater progress in TIMSS. 


Science — year 5 


Year 5 pupils’ performance in science has been consistently and significantly above the 
TIMSS centrepoint in all cycles (see Figure 5). England’s performance in 2019 has 
improved since 2011 when there was a significant decline'*. The performance of year 5 
pupils remained broadly similar in 2019 compared with 2015. 


Figure 5: Trend in the average year 5 science score (England) 
Average 
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Source: TIMSS 2019. 


Note 1: The 1999 cycle of TIMSS included only year 9 pupils, represented by the dashed 
line. 

Note 2: The 1995 score is an average across the performance of year 4 and year 5 
pupils as the 1995 cycle assessed pupils across both year groups. 

Note 3: Response rates for TIMSS in England were relatively low in 1995, 1999 and 
2003. 

Note 4: Scores that represent a significant increase or decrease from the previous 
TIMSS cycle are marked with an asterisk (*). 


12The 1995 score was an average across the performance of year 4 and year 5 pupils. 
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Six countries (Chinese Taipei, Finland, Japan, the Republic of Korea, Russia and 
Singapore) performed significantly above England in science in year 5; this was 3 fewer 
than in 2015. Nine performed at a similar level to England and 42 significantly below. 
Figure 6 shows that in 2019, the percentage of year 5 pupils reaching each benchmark 
remained relatively stable since 2015. Over time, the percentage of pupils reaching the 
advanced benchmark has shown a significant decline, although the percentage reaching 
the high benchmark or above has been more stable. By contrast, the percentages of 
pupils reaching the intermediate and low benchmarks or above show significant 
improvement over recent TIMSS cycles. 


Figure 6: Trend in the percentage of year 5 pupils reaching each of the TIMSS 
international benchmarks in science (England) 
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Source: TIMSS 2019. 
Note 1: The 1999 cycle of TIMSS included only year 9 pupils. 
Note 2: The 1995 score is an average across the performance of year 4 and year 5 
pupils as the 1995 cycle assessed pupils across both year groups. 
Note 3: Response rates for TIMSS in England were relatively low in 1995, 1999 and 
2003. 


In 2019, the difference between England’s highest- and lowest-performing year 5 pupils 


in science was 236 scale points, the median difference for the comparator group 
countries. 
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Science — year 9 


The performance of year 9 pupils in England in science decreased in 2019, meaning that 
it was significantly below all previous TIMSS cycles (see Figure 7 below). However, the 
average scores of year 9 pupils still remained significantly above the TIMSS centrepoint. 


Figure 7: Trend in average year 9 science score (England) 
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Source: TIMSS 2019. 


Note 1: The 1995 score is an average across the performance of year 8 and year 9 
pupils as the 1995 cycle assessed pupils across both year groups. 

Note 2: Response rates for TIMSS in England were relatively low in 1995, 1999 and 
2003. 

Note 3: Scores that represent a significant increase or decrease from the previous 
TIMSS cycle are marked with an asterisk (*). 


Nine countries performed significantly above England, 4 more than in 2015. These were 
Australia, Chinese Taipei, Finland, Hungary, Japan, Lithuania, the Republic of Korea, 
Russia and Singapore. Seven performed at a similar level and 22 significantly below. 


A larger percentage of year 9 pupils achieved each of the international benchmarks in 
England compared with the international median across all participating countries (see 
Figure 8). However, the percentage of pupils performing below the low benchmark (11%) 
was more than double that of 2015 (5%) and the percentages of pupils reaching the high, 
intermediate and low benchmarks or above were all significantly below those recorded in 
2015. 
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Figure 8: Trend in the percentage of year 9 pupils reaching each of the TIMSS 
international benchmarks in science (England) 
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Source: TIMSS 2019. 


Note 1: The 1995 score is an average across the performance of year 8 and year 9 
pupils as the 1995 cycle assessed pupils across both year groups. 

Note 2: Response rates for TIMSS in England were relatively low in 1995, 1999 and 
2003. 


In 2019, England had a relatively large difference between its highest- and lowest- 
performing year 9 pupils (a range of 302 scale points) in science. At year 9 only New 
Zealand and the United States had a larger range of science scores. 


TIMSS allows for a comparison of a cohort’s performance over 2 cycles as year 9 pupils 
in 2019 were in year 5 in 2015'°. Relative to the TIMSS centrepoint, this cohort of pupils 
performed better in year 5 science than in year 9 science. A similar decline in relative 
performance against the centrepoint was reported in most of the comparator countries 
except Australia, Chinese Taipei, France, Japan, Lithuania and Singapore, all of which 
reported higher relative performance in year 9 than in year 5. 


13 Although the 2015 year 5 pupils and 2019 year 9 pupils were from the same cohort, different pupils from 
the cohort were selected to take part in TIMSS in each cycle. 
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Pupils’ performance in the content and cognitive domains 


In both mathematics and science, pupils in England in 2019 performed above the 
international averages in all content domains (apart from algebra for year 9) for both year 
5 and year 9. 


In mathematics in 2019, the performance of year 5 pupils was strongest in the data and 
number domains, both of which improved significantly from 2015; they were weakest in 
measurement and geometry, in which performance was relatively stable compared to 
2015. The relative strength in data was in contrast to the majority of the highest- 
performing countries, which tended to perform strongly in measurement and geometry. 
Pupils in England were strongest in the knowing cognitive domain, as they were in 2015. 


Year 9 pupils were strongest in data and probability and in number, and weakest in both 
algebra and geometry. These relative strengths and weaknesses mirrored the 2015 
outcomes. The strengths of pupils in the highest-performing countries tended to lie 
across the algebra and geometry domains. Year 9 pupils in England were strongest in 
the applying cognitive domain, whereas in 2015 they had been strongest in the reasoning 
domain. 


In science in 2019, year 5 pupils’ performance in the physical and life science domains 
was the same as England’s overall science average score, and their performance was 
weakest in Earth science. Performance in 2019 was similar to 2015 across all the content 
domains. Year 5 pupils’ performance in England was strongest in the knowing and 
reasoning cognitive domains and weakest in the applying domain. In 2015, there were no 
significant differences between average scores for each cognitive domain and the overall 
science average score. 


In 2019, the performance of year 9 pupils across all content domains (biology, chemistry, 
physics and Earth science) was in line with England’s overall science average score: that 
is, there were no notable strengths or weaknesses. Year 9 pupils’ average scores in all 
content domains were significantly lower in 2019 than in 2015, reflecting England’s lower 
overall year 9 science average in 2019. Pupils’ performance in England in the knowing 
and applying cognitive domains was similar to the overall average scale score. In the 
reasoning domain pupils scored significantly below the overall average. This was in 
contrast to 2015, when reasoning was the strongest cognitive domain for year 9 pupils. 


Differences by pupil characteristics 


In 2019 there were no significant differences between the performances of boys and girls 
across either subject or year group. In 2015 year 5 boys significantly outperformed girls 
in mathematics. 


Performance across all ethnic groups did not differ significantly in mathematics in either 
year 5 or year 9. In science in both year 5 and 9 pupils from ethnic groups aside from 
Black pupils performed comparably to White British pupils; Black pupils scored 
significantly below them. 


Performance by pupils for whom English was not their first language did not differ 
significantly in mathematics in either year 5 or year 9 and in science in year 5. In year 9 
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science pupils whose first language was English significantly outperformed pupils for 
whom English was not a first language. 


Pupils who had been eligible for free school meals (FSM) at any time in the last 6 years 
performed significantly lower than their non-eligible peers across both year groups and 
both subjects. TIMSS asks participating pupils how many books they have at home, as a 
proxy for socio-economic status. This measure revealed a wide gap in performance for 
both year groups and in both subjects between pupils who had very few books at home 
and those who had many. The achievement gap between the 2 groups was around 100 
scale points across both cohorts and subjects. 


Overall, with a few exceptions, there were generally no significant differences by pupil 
characteristics for reaching the international benchmarks. The main exception was for 
FSM pupils (as well as for pupils with few books at home), of whom a significantly lower 
percentage reached each benchmark in either year group and either subject. 


In year 5 science, significantly fewer Asian pupils reached the advanced benchmark than 
White British pupils, while in year 9 science, significantly fewer pupils whose first 
language was not English reached the intermediate and high benchmarks or above than 
White British pupils. In year 9 science significantly fewer Black pupils reached the low 
benchmark or above than White British pupils. 


Pupil attitudes and aspirations in mathematics and science 


Overall, analysis indicated that pupil confidence and, to a lesser extent, a liking for 
learning a subject were strongly associated with achievement, with pupils’ reports of their 
lessons’ instructional clarity and valuing the subject less strongly associated. It is 
important to note that an association (or correlation) between 2 variables (such as level 
of confidence and average achievement) is not the same as causation (i.e. that one thing 
causes the other). 


There was a positive and significant association between pupils’ confidence in their 
mathematical ability and their average achievement — in both years 5 and 9 very 
confident pupils scored 100 scale points higher than pupils who were not confident. 
Although the scale point difference was not as high, the same associations were evident 
for science in years 5 and 9. 


Pupils in both year groups who liked learning mathematics and science scored higher, on 
average, than those who did not like learning those subjects. Year 9'4 pupils who valued 
mathematics and science strongly scored higher, on average, than their peers who did 
not value those subjects. 


There was also a positive and significant association for both year 5 and year 9 pupils 
between mathematics achievement and the extent to which they reported that lessons 
provided instructional clarity; the same was evident for year 9, but not year 5, science 
and between mathematics achievement and the extent to which they agreed that they 
would like to study mathematics after secondary school. 


14 Only year 9 pupils in England were asked how much they valued mathematics and science. 
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In both year groups and subjects, boys were significantly more confident, and liked 
learning more than girls. Boys also valued both subjects more in year 9. Significantly 
more girls than boys were not confident in their mathematical or scientific ability, and 
reported that they did not like mathematics or science. 


School environment and resources 


The 3 school-level factors most strongly associated with pupils’ achievement at both 
years 5 and 9 in England were an emphasis on academic success (positive association), 
disorderly behaviour in school (negative association) and experiences of bullying 
behaviour in schools (negative association). It is important to note that an association 
between 2 variables is not the same as causation. 


In both subjects and for both year groups there was a positive and significant association 
between schools that placed an emphasis on academic success (as reported by 
headteachers) and achievement; the greater the emphasis on academic success, the 
higher the pupils’ achievement. 


Responses to the perceptions of discipline, school safety and orderliness and bullying 
revealed a negative association with pupils’ average achievement: the less that pupils 
were adversely impacted, the higher their average achievement. Resource shortages 
were negatively associated with average achievement only for year 5 pupils in science. 
Headteachers reported few discipline problems for a majority of year 5 and year 9 pupils, 
but pupils taught in schools with discipline problems achieved less well on average than 
those taught in schools without such problems. In both years 5 and 9, there were no 
pupils taught in schools where headteachers reported moderate to severe discipline 
problems. A majority of pupils were taught in schools reported by teachers to be safe and 
orderly. 


Most year 5 and year 9 pupils reported that they never or almost never experienced 
bullying behaviours. There was a negative and significant association between the extent 
to which pupils experienced bullying behaviours and their average achievement in 
England, with pupils who reported experiencing bullying more frequently having 
significantly lower average achievement. There was also a negative association between 
average achievement and the extent to which pupils reported experiencing disorderly 
behaviour in lessons. 
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Teachers and teaching 


Year 5 and 9 pupils in England were more likely to be taught by teachers with fewer 
years of experience than the average across most of the comparator countries. Year 9 
pupils taught by teachers with 20 or more years’ experience had significantly higher 
mathematics scores than those pupils taught by teachers with fewer than 5 years’ 
experience. However, length of teacher experience was not associated with higher 
scores for year 5 pupils or higher science scores for year 9. 


Around a quarter of year 5 pupils were taught by teachers with mathematics or science 
as either their main area of study or joint area of study with primary education. Pupils’ 
average mathematics scores were not significantly different depending on their year 5 
teachers’ specialisms; for science, year 5 pupils taught by teachers with science but not 
primary education as their main area of study had higher average achievement than 
other pupils. Similarly, pupils taught by teachers with a main area of study in primary 
education but not science had significantly higher average scores than pupils taught by 
teachers with both science and primary education as main areas of study or by teachers 
with other main areas of study. 


The majority of year 9 pupils in England were taught mathematics by teachers who had 
studied mathematics as either their main area of study or a joint main area of study with 
mathematics education, and science by teachers who had studied science as either their 
main area of study or a joint main area of study with science education. 


When asked about their continuing professional development (CPD) needs, year 9 
teachers in both subjects highlighted the need for more support to improve pupils’ critical 
thinking or problem-solving skills, and integration of technology into their teaching 
practice. 


There was a positive association between teachers’ reported job satisfaction and year 9 
pupils’ average mathematics scores — the average score for England’s pupils taught by 
teachers who reported being very satisfied with their job was significantly above the 
score for pupils whose teachers were less satisfied with their job. However there were no 
associations between teachers’ job satisfaction and pupils’ achievement in mathematics 
in year 5, or in science in either year 5 or year 9. 


In 2019 the majority of year 9 pupils in England spent between 16 and 30 minutes per 
subject each week on mathematics and science homework tasks ‘>. There was a positive 
association between pupils spending between 31 and 60 minutes on homework and 
higher average achievement, but no association when pupils spent more than an hour. 


Year 9 pupils in England who did and did not have access to computers during science 
and mathematics lessons had average scores that were almost the same. The frequency 
with which pupils were set tests on computers was not associated with any significant 
differences in average mathematics scores, but there was a positive association between 
year 9 pupils taking science tests on computers once a month or more and higher 
average scores compared to peers who took such tests only once or twice a year. 


15 Only year 9 pupils in England were asked about homework tasks. 
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Home environment 


The majority of year 9'° pupils reported that they had some home resources for learning. 
Year 9 pupils with many resources for home learning had higher mathematics and 
science scores on average than peers with fewer such resources. 


The vast majority of year 9 pupils in England reported having access to the internet at 
home and the majority of both year 5 and year 9 pupils in England had their own 
computer/tablet and/or access to their own mobile phones, with more year 9 pupils 
having access than year 5 pupils. Most year 9 pupils in England used a computer for 
their homework. 


The majority of both year 5 and year 9 pupils in England had access to a study desk at 
home, with more year 9 pupils having access than year 5 pupils. 


The uptake of additional tuition in mathematics and science by year 9 pupils in England 
was low compared to other countries, and pupils who received tuition performed 
significantly less well than pupils who did not'”. The 6 highest-performing countries had 
the largest percentages of pupils receiving tuition in both subjects. 


In 2019, most year 5 and year 9 pupils in England reported that they were never or 
almost never absent from school. For year 5 and year 9 pupils there was a positive 
association between lower absence rates and higher achievement in both mathematics 
and science. 


Conclusions 


From a relatively poor performance in mathematics in both age groups in 1995, 
performance in mathematics has significantly improved in both years 5 and 9'8. This was 
particularly true for year 5, where England’s average score in 2019 was the highest of 
any TIMSS cycle. At year 9 performance in mathematics has been relatively stable since 
2007. 


In science, performance for year 5 pupils has exhibited significant improvement over the 
last 24 years, and remained broadly similar in 2019 compared with 2015. For year 9 
pupils the picture was somewhat different. Whereas over the first 20 years of TIMSS 
performance by England’s year 9 pupils in science had been stable, in 2019 it dropped 
significantly, and was significantly lower than in any previous TIMSS cycle. The 
percentage of pupils performing below the low benchmark also doubled compared to 
2015. The reasons for this change are not obvious and require further research. 


Although pupils from a group of mostly East Asian countries — Chinese Taipei, Hong 
Kong, Japan, Republic of Korea, Russia, Singapore — have consistently outperformed 


18 In England only year 9 pupils were asked about home resources, attendance at additional tuition and use 
of the internet for homework. 

1? Caution should be used in interpreting the relationship, since pupils might attend additional tuition based 
on their relatively low prior academic performance. 

18 The 1995 score was an average across the performance of 2 year groups (year 4 and year 5 pupils; year 
8 and year 9 pupils). 
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England’s pupils, we must not lose sight of the fact that pupils in England did consistently 
well against the international average in both subjects and in both cohorts. England’s 
pupils also fared well when compared to their counterparts in other English-speaking 
countries as well as compared to pupils from a representative group of other European 
countries. 


Aside from year 9 science, performance issues were similar to those highlighted in 2015: 
between years 5 and 9 pupils’ scores did not increase and, at times, scores decreased. 
Fewer pupils in England reached the advanced and high benchmarks than those in the 
highest-performing countries. Wide achievement gaps also remained between England’s 
most and least advantaged pupils. 


Performance in different domains of mathematics was either stable or improved from 
2015 — for year 5 in the data and number domains the improvement was significant. This 
relative stability in both the content and cognitive domains was also true for year 5 
science. The most notable performance issue in 2019 was in year 9 science, where 
pupils’ performance was significantly weaker in all content and cognitive domains than it 
had been in 2015. In 2019 pupils were weakest in the reasoning domain, in contrast to 
2015 when this was the strongest cognitive domain. 


Gender differences were clear in responses to questions asked about confidence in 
mathematics and science as well as liking for the subjects. While overall the more 
confident pupils were and the more they enjoyed the subject, the better they performed in 
it, girls were significantly less confident and liked the subject less in both year groups and 
for both subjects. These negative aspects did not manifest themselves in differences in 
achievement, however, with girls’ outcomes not significantly different from boys’. Pupils 
from disadvantaged backgrounds, as in TIMSS 2015, performed less well than their more 
advantaged peers. Black pupils made some gains in 2019 compared to 2015 but some 
gaps between Black and White British pupils’ performance remained. On the whole there 
were no significant differences in scores for pupils with and without English as a first 
language. 


Both mathematics and science teachers saw their greatest professional development 
needs in the areas of incorporating technology into teaching as well as including problem 
solving and critical thinking in lessons. Where headteachers reported that their school 
policies were orientated towards academic performance this focus was reflected in 
stronger overall pupil performance. No pupils in the study had teachers who reported that 
their schools experienced moderate to severe discipline problems, and pupils' survey 
responses largely echoed this finding. Where pupils did indicate evidence of frequent 
bullying and/or disruption in their classrooms, their performance was lower than for those 
who did not report these behaviours. 


Overall the 2019 TIMSS results saw an improvement in year 5 pupils’ performance in 
mathematics, stability in year 9 mathematics and year 5 science and a decline in year 9 
performance in science. This would indicate the need for research to investigate the year 
9 science outcomes, possibly looking at PISA science outcomes too. TIMSS data could 
inform the myriad research already underway into why girls lack confidence in and 
enjoyment of mathematics and science and may hold some useful findings for 
researchers studying behavioural issues such as bullying. 
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Chapter 1. Introduction 


1.1 What is the Trends in International Mathematics and 
Science Study (TIMSS)? 


Designed by the International Association for the Evaluation of Educational Achievement 
(IEA), TIMSS is a worldwide research project that takes place every 4 years'®. Boston 
College in the United States coordinates TIMSS with support from the IEA, Statistics 
Canada and the Educational Testing Service. TIMSS is 1of 3 international large-scale 
assessments (ILSAs) described in section 1.2.3 below. 


The study’s main purpose is to provide internationally comparable data about trends in 
pupils’ mathematics and science achievement at primary and secondary school levels 
over time. Teachers and headteachers in participating schools complete questionnaires 
on factors that potentially have an impact on academic achievement. The findings from 
TIMSS can therefore have policy and practice implications for readers. Pupil data are 
collected through academic assessments and attitudinal surveys. Contextual data from 
the pupils’ headteachers and teachers are also collected through attitudinal surveys. 


TIMSS was first carried out in 1995 and data have been collected every 4 years since, so 
that 2019 represents the study's 7th cycle over a 24-year period*°. To enable robust 
international comparisons, the study uses data collected from samples of pupils in the 
same academic year groups: pupils aged 9-10 and 13-14. In England, these pupils are 
in years 5 and 92". 


In 2019, 64 countries and 8 benchmarking systems (states and provinces within 
countries that collect representative samples in TIMSS and so can provide comparative 
findings) participated in TIMSS (see Table 1 below). Across these countries and systems, 
more than 580,000 pupils participated in 2019. Information about the study design and 
conduct in each country can be found in the T/MSS International Report 201972. In 

2019, 58 countries and 6 benchmarking systems participated in the 4" grade (year 5 in 
England) TIMSS and 39 countries and 7 benchmarking systems participated in the 8" 
grade (year 9 in England TIMSS. England participated in both the year 5 and 9 
mathematics and science assessments in 2019 and has participated since 1995. 


18 The IEA (International Association for the Evaluation of Educational Achievement) ‘is an international 
cooperative of national research institutions, governmental research agencies, scholars, and analysts 
working to research, understand, and improve education worldwide.’ It conducts ‘high-quality, large-scale 
comparative studies of education across the globe in order to provide educators, policymakers, and parents 
with insights into how students perform’ (source: httos://www.iea.nl/). Its list of member states is available 
at:_https://www.iea.nl/about/members/institutional 

20 The 1999 study in England included year 9 pupils only. 

21In the IEA’s methodology and TIMSS International Reports, these year groups are referred to as 4" and 
8" grade, reflecting terminology used across the range of participating countries. 

22 Available at https://timssandpirls.bc.edu/timss2019/ 
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Table 1: TIMSS 2019: participating countries and benchmarking systems 


Benchmarking 
systems 


Continents, Participating countries and benchmarking systems 
regions and 
systems 

Africa Egypt, Morocco, South Africa 

Asia Chinese Taipei, Hong Kong SAR23, Japan, Kazakhstan, Malaysia, 
Pakistan, Philippines, Republic of Korea, Russian Federation24, 
Singapore, Turkey 

Australasia Australia, New Zealand 

Europe Albania, Armenia, Austria, Azerbaijan, Belgium (Flemish), Bosnia and 
Herzegovina, Bulgaria, Croatia, Cyprus, Czech Republic, Denmark, 
England, Finland, France, Georgia, Germany, Hungary, Ireland, Italy, 
Kosovo, Latvia, Lithuania, Malta, Montenegro, Netherlands, Northern 
Ireland, Norway, Poland, Portugal, Republic of North Macedonia, 
Romania, Serbia, Slovak Republic, Spain, Sweden 

The Middle Bahrain, Iran, Israel, Jordan, Kuwait, Lebanon, 

East Oman, Qatar, Saudi Arabia, United Arab Emirates (UAE) 

The Canada, Chile, United States 

Americas 


Abu Dhabi (UAE), Dubai (UAE), Gauteng (South Africa), Madrid 
(Spain), Moscow (Russian Federation), Ontario (Canada), Quebec 
(Canada), Western Cape (South Africa) 


A consortium comprising Pearson and the UCL Institute of Education (UCL IOE) 
managed test administration, national data analysis and reporting in England. Pearson 
recruited schools for the field trial and main study assessments, adapted the test items 
for use in England and supported participating schools in the administration of the tests 
during the main study period from February 25" to June 13" 2019. Pearson also marked 
all assessment and questionnaire responses and undertook a curriculum matching 
exercise to identify which of the TIMSS test items pupils in English schools would have 
been expected to have studied by the time they took the TIMSS assessments. The UCL 
IOE team was responsible for national data analysis and the writing of this national 


report. 


The IEA analysed the international database of country results and the evidence from 
pupil, headteacher and teacher questionnaires. This analysis is available in the IEA’s 
TIMSS International Report 2019. The IEA also commissioned a TIMSS Encyclopedia?° 
chapter from each participating country to provide an overview of the structure of each 
participating education system; England’s chapter was written by the UCL IOE team. 


23 Hong Kong Special Administrative Region (SAR) is referred to as Hong Kong in the report. 
4 Russian Federation is referred to as Russia from here on in the report. 
28 The TIMSS 2019 Encyclopedia: Education Policy and Curriculum in Mathematics and Science. 
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Appendix B provides more detailed information about the TIMSS survey methodology 
and the processes that underpinned the creation of the IEA’s TIMSS International Report 
2019. 


1.2 What TIMSS tells us 


1.2.1 Why England participates in TIMSS 


TIMSS enables governments to benchmark education policy and performance, to make 
evidence-based decisions and to learn from one another. Participation is also of great 
value to academic and research communities and to participating schools. 

In England, TIMSS gives interested individuals and organisations important insights into 
how well pupils are performing in mathematics and science in years 5 and 9 at the 
content and cognitive levels, in relation both to England’s previous achievements — 
trends over time — and to the achievements of pupils in other participating countries. 
TIMSS also provides a valuable opportunity for achievement to be considered in the 
context of school and background factors that potentially influence achievement. The 
factors reported in the study include: 


e pupils’ attitudes towards mathematics and science 

e pupils’ perceptions of teaching in these subjects 

e teachers’ education, experience and job satisfaction 

e headteachers’ and teachers’ views on school discipline and resources; and 


e pupils’ reports on their home environment and resources at home 


1.2.2 What is the impact of TIMSS? 


England has taken part in all TIMSS cycles since 1995. Policymakers, educators, 
academics and research organisations in England study the results to explore the 
potential for improvements in teaching, learning and assessment of mathematics and 
science, and to conduct further research relating to significant changes in results. Factors 
of interest include the proportion of pupils reaching each international benchmark and the 
range of scores — with particular interest in narrowing achievement gaps between 
advantaged and disadvantaged pupils. Using matched data from England’s National 
Pupil Database provides insights into potential relationships between pupils’ achievement 
and characteristics such as gender, ethnicity, socio-economic status and first language. 
Comparisons can be made between how much pupils value learning mathematics and 
science and their TIMSS performance. Awareness of teachers’ and headteachers’ 
perceptions of the availability of school resources and professional development 
opportunities can provide evidence to guide suggested areas for future planning. 


Since 1995, TIMSS findings (together with those from other international benchmark 
studies) have been used to identify priorities for improving mathematics and science 
policy and practice — for example, informing the activities of the National Centre for 
Excellence in the Teaching of Mathematics (NCETM)*6 and the National Science, 


26 See https://www.ncetm.org.uk/ 
23 


Technology, Engineering, and Mathematics (STEM) Learning Centre?’. 


1.2.3 How does TIMSS compare to other international surveys? 


England takes part in 2 other international large-scale assessments looking at the 
performance of pupils in schools: the Progress in International Reading Literacy Study 
(PIRLS) and the Programme for International Student Assessment (PISA). 


The Progress in International Reading Literacy Study (PIRLS) programme is organised in 
a similar manner to TIMSS. PIRLS is also coordinated by the IEA and is an international 
test for pupils in the 4'" grade (year 5 in England) that measures pupils’ reading literacy. It 
is administered every 5 years. The first iteration took place in 2001 and England has 
participated in every cycle. Like TIMSS, the PIRLS assessments survey teachers and 
headteachers to document school and teacher instructional practices and other school 
experiences related to developing reading literacy. Pupils also complete questionnaires 
about their attitudes toward reading and their reading habits. The most recent cycle, in 
2016, included 61 countries and benchmarking systems, and introduced an online 
assessment of reading called ePIRLS. Pupils in England scored significantly above the 
international centrepoint, with a score below that of 7 countries, similar to that of 6 and 
significantly above that of 362°. 


The curriculum model in TIMSS differs from that used in the Programme for International 
Student Assessment (PISA) study from the Organisation for Economic Co-operation and 
Development (OECD), which was last administered in 2018. This 3-yearly international 
study assesses pupils aged 15 (primarily in year 11 in England) in reading, mathematics 
and science. TIMSS and PISA are complementary, but differ in particular ways: TIMSS 
assesses pupils across 2 separate year groups and its assessments are focused on 
pupils’ knowledge and understanding of curriculum content, whereas PISA assesses the 
application of education to real-life problems in reading, mathematics and science 
literacy. In 2018, 79 countries participated in PISA. 


In PISA 2018, mean scores in England were significantly above the OECD average in all 
3 subjects (reading 505, OECD average 487; science 507, OECD average 489; 
mathematics 504, OECD average 489). England’s mean scores in reading and science 
have not changed significantly over successive PISA cycles, but in mathematics, 
England’s mean score showed a statistically significant increase in comparison with PISA 
2015”. 


Please see this report's conclusion for further discussion on TIMSS and PISA 
performance in England. 


27 See https://www.stem.org.uk/ 
8 Progress in International Reading Literacy Study (PIRLS): National Report for England (2017). Retrieved 


from: _https://www.gov.uk/government/publications/pirls-2016-reading-literacy-performance-in-england 
29 Achievement of 15- year-olds in England: PISA 2018 results (2019). Programme for International Student 


Assessment (PISA) National Report for England (2018). Retrieved from: 
https://www.gov.uk/government/publications/pisa-2018-national-report-for-england 
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1.3 About the TIMSS sample 


All countries and benchmarking systems participating in TIMSS followed strict guidelines 
and sampling targets to ensure that the group of pupils that eventually participated in the 
study was nationally representative. 


In England, 150 primary and 151 secondary schools were invited to participate in the 
main TIMSS study. Schools were selected according to a sampling framework 
representative of all schools in England. Depending on class size, 1 or 2 randomly 
selected year 5 or year 9 classes were chosen from each participating school and all the 
pupils from the selected classes were asked to participate in the study*°. 


The IEA’s sampling referee inspected the school and pupil samples, and they were 
accepted for TIMSS 2019 if they met 1 or both of the following criteria: 


e aminimum school participation rate of 85% 


e aminimum combined school, classroom and student participation rate of 75%, based 
on main sample schools (although classroom and student participation rates include 
replacement schools) 


In England, a total of 3,396 year 5 pupils from 139 primary schools participated in TIMSS 
2019, an 86% main-sample school participation rate, exceeding the first participation 
criterion. A total of 3,365 year 9 pupils from 136 secondary schools participated in TIMSS 
2019, an 83% main-sample school participation rate, not meeting the first participation 
criterion. However, a 79% overall participation rate was achieved, exceeding the second 
IEA criterion. 


Tables 2 and 3 below summarise the characteristics of the TIMSS school and pupil 
samples for England in 2019 and demonstrate that England’s year 5 and year 9 samples 
were representative of primary and secondary schools nationally. 


Table 2: Schools participating in TIMSS (England, 2019) 


Mainstream Mainstream 
Neale rima cate seconda 
School type TIMSS seellal | TIMSS ry 
ear eie schools eaple schools 
- (England) - (England) 
TIMSS sample schools 139 - 136 - 
Independent schools 8 - 8 - 
State-funded schools 131 16,769 128 3,448 
Academy schools 33.6% 30.9% 70.3% 67.1% 
Community schools 42.0% 37.4% 14.0% 11.5% 
Foundation schools 0.7% 3.4% 6.3% 5.4% 


30 A small number of pupils were excluded from the tests. See Appendix B for further details about the 
sampling methodology. 
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Mainstream Mainstream 
veat5 rima wedi seconda 
School type TIMSS ieiees TIMSS ry 
exanle schools esnIe schools 
“ (England) y (England) 
Voluntary aided schools | 16.0% 16.6% 6.3% 7.0% 
| 
Voluntary controlled 5 9% 10.7% 0.7% 1.0% 
schools 
hool T 
pISe SeNOC ee ECS ANG lino) 1.0% 2.3% 7.9% 
studio schools 


Source: TIMSS 2019, National Pupil Database 2019, School Census 2019*' 


Note 1: National data for independent schools is not disaggregated by phase or 
mainstream / special school 


Table 3a: Pupils participating in TIMSS (England, 2019) 


Pupils in Pupils in 
Years mainstream, Year 9 mainstream, 
: ae state funded state-funded 
Pupil characteristics TIMSS primary TIMSS secondary 
sample sample 
schools schools 
(England) (England) 
Total number of pupils in 
TIMSS 3,396 - 3,365 - 
Number of pupils with a 
national pupil database | 3,214 - 3,194 - 
record 
oe eu 49.9% 50.9% 47.0% 50.7% 
ala offemale | 50.1% 49.1% 53.0% 49.3% 
Percentage of pupils 
eligible for free school 6 5 é és 
meals (FSM) in the last 27.1% 23.2% 26.2% 28.1% 
6 years 
Percentage of pupils for 
whom English is not 19.9% 21.2% 16.3% 16.9% 
their first language 


Source: TIMSS 2019, National Pupil Database 2019, School Census 2019*4 


31 https:/Awww.gov.uk/government/statistics/schools-pupils-and-their-characteristics-january-2019 
32 https:/Awww.gov.uk/government/statistics/schools-pupils-and-their-characteristics-january-2019 
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Note: Pupil profile data is presented for TIMSS pupils with a national pupil database 
record. Percentages may not sum to 100% due to rounding. 


Table 4b: Pupils participating in TIMSS (England, 2019): ethnicity 


Pupils in Pupils in 
mainstream, mainstream, 
: ee Weer state funded vice state-funded 
Pupil characteristics TIMSS ‘ TIMSS 
sample primary sample secondary 
schools schools 
(England) (England) 
White British 67.4% 65.5% 71.2% 67.0% 
White Other 8.9% 8.1% 5.4% 6.2% 
Asian 7.2% 11.2% 11.4% 11.3% 
Mixed 6.5% 6.3% 4.7% 5.5% 
Black 5.9% 5.5% 4.9% 6.0% 
Other 3.5% 2.0% 2.2% 1.9% 
Chinese 0.6% 0.5% 0.3% 0.4% 


Source: TIMSS 2019, National Pupil Database 2019, School Census 2019% 


Note 1: Nationally a small percentage of pupils did not have ethnicity data. 
Note 2: Pupil profile data is presented for TIMSS pupils with a national pupil database 
record. Percentages may not sum to 100% due to rounding. 


1.4 Report structure 


This report is structured using a series of questions that were asked of the TIMSS 

2019 data. These enable users to identify the questions most relevant to them. Data for 
England in 2019 are presented for each question and comparisons made, as appropriate, 
with previous TIMSS studies and/or other countries’ data. England’s TIMSS data 

have also been matched to data from the National Pupil Database (NPD), allowing 
additional analysis of factors such as free school meals (FSM), ethnicity and English as 
an additional language that would not have been possible using TIMSS data alone. 


The report comprises 6 main foci: 


1. Overall performance in mathematics and science. This section (chapters 3-5) focuses 
on how England’s year 5 and 9 pupils have performed over time, and in comparison with 
other countries, both in terms of average achievement and achievement against 
international benchmarks. It includes analyses of how pupils have performed in different 
aspects of the curriculum (content domains), as well as in different cognitive domains. 


33 https:/Awww.gov.uk/government/statistics/schools-pupils-and-their-characteristics-january-2019 
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2. Differences in mathematics and science performance by pupil characteristics. This 
section (chapter 6) focuses on how well different groups of England’s year 5 and 9 pupils 
have performed in comparison to each other and, where appropriate, with other 
countries. 


3. Pupil engagement and confidence in mathematics and science. This section (chapter 
7) focuses on pupils’ attitudes towards their teaching, their subject confidence and 
whether they like and value mathematics and science, compared with pupils in other 
countries. 


4. School environment and resources. This section (chapter 8) considers whole-school 
issues, such as the extent to which schools focus on academic success, to provide a 
broader context to the schooling that England’s year 5 and 9 pupils receive, and to 
consider how this compares to their peers in other countries. 


5. Teachers and teaching. This section (chapter 9) focuses on matters such as teachers’ 
professional development, years of teaching experience and the use of computers in the 
classroom. Where appropriate, the chapter makes comparisons with other countries. 


6. Home environment. This section (chapter 10) focuses on the extent to which England’s 
year 5 and 9 pupils are supported in their mathematics and science learning through 
resources at home and how they use these. It also focuses on the extent to which they 
attended additional tuition, for what purpose and its impact on achievement. 
Comparisons are provided with the experiences of pupils in other countries. 


The conclusion draws together the main findings and provides some reflections upon 
their implications for policy and practice in England. 
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1.5 Comparator countries 


Throughout the report, comparisons are made with other countries that took part in the 
study. The report analyses England’s performance in relation to all participating countries 
in some places, but readers are generally referred to the IEA’s TIMSS International 
Report 2019 for such comparisons. 


Analysis in this report focuses on England’s performance compared to a sub-set of 
participating countries; these were selected to provide relevant and interesting 
comparisons. 


The comparator countries referenced in this report fit into one or more of the following 
categories: 


e highest-performing countries that over time have consistently performed 
significantly better than England in TIMSS (6 countries: Chinese Taipei, Hong Kong, 
Japan, Republic of Korea, Russia, Singapore) 


e other English-speaking countries, since these can be seen as having similar 
contexts to England and provide helpful benchmarks for TIMSS (6 countries: 
Australia, Canada, Ireland, New Zealand, Northern Ireland, United States) 


e aselection of European countries, chosen to provide a comprehensive view of 
performance across Europe in relation to TIMSS (10 countries: Finland, France, 
Germany, Italy, Lithuania, Netherlands, Norway, Poland, Spain, Sweden) 


Whenever comparisons are made with other countries it is important to consider the 
potential effect of cultural differences. This is particularly important in chapters 7— 

10, which draw on responses from the attitudinal questionnaires that accompanied the 
main TIMSS assessments“. 


Although the benchmarking systems follow the same guidelines that apply to countries 
participating in TIMSS, in this report international comparisons are made between 
England and other participating countries, rather than with these systems. 


1.6 Interpreting differences over time and between countries 


Throughout the report, explanations of how the data were collected are given so that 
users can understand the methodology used and how to interpret data presented. Where 
the terms ‘significant’ or ‘not significant’ are given, these mean that the finding referred to 
is either statistically significant or not statistically significant at conventional levels*°. 


34 The TIMSS process involves a rigorous translation and cultural adaptation phase during which 

the wording of questions is tested for differential item functioning (DIF) according to culture and language. 
DIF refers to group differences in performances on a test question (item) amongst test-takers who are 
comparable in terms of their overall proficiency. 

35 Five per cent significance tests are applied throughout. Significance levels will depend on the averages 
but also on the standard errors. Both averages and standard errors are used to calculate a T-statistic which 
is then compared to the critical values in t-tables. 
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In order to understand which interpretations and conclusions can reasonably be drawn 
from the TIMSS data, it is important to keep factors such as sampling error and 
measurement error in mind. No test results can be entirely free from error, and error 
needs to be understood in its technical sense in the context of this report. 


Sampling error arises because the statistical characteristics of a population as a whole 
must be estimated using a subset, or sample, of that population. A different sample for 
England’s population might produce slightly different results. Only if every year 5 and 
year 9 pupil in England (the entire population) had taken part in TIMSS assessments 
could the outcomes be interpreted as totally consistent and representative. TIMSS 
sampling methodology** — which makes use of the jackknife repeated replication (JRR) — 
is derived to minimise sampling error, but it cannot entirely eliminate it, which is why 
confidence intervals and standard error measurements are included in TIMSS reports’. 


The same holds true for measurement error, which can occur when test instruments do 
not accurately measure the knowledge or aptitude they are meant to measure. In TIMSS 
assessments, a potential source of this error comes from the different curricula in 
participating countries. As with sampling error, the TIMSS methodology attempts to offset 
measurement error by using the Test-Curriculum Matching Analysis, in which each 
participating country identifies, for each item, whether or not the topic is found in the 
curriculum for the majority of its pupils*®. 


These 2 factors offer useful background to understanding TIMSS rank ordering and 
differences in scores over time. This is the reason this study concentrates on statistically 
significant differences rather than reporting on simple rank orders or score changes. 
Significant differences are less likely to have been caused by sampling or measurement 
errors. It is also important to remember that changes in ranking over time may result from 
changes to the cohort of countries participating in each cycle. 


36 See https://timssandpirls.bc.edu/timss2019/methods/index.html 
3” See https://nces.ed.gov/statprog/handbook/timss dataquality.asp 
38 See https://nces.ed.gov/statprog/handbook/timss dataquality.asp 
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Chapter 2. TIMSS assessment approach and 
curriculum match 


The TIMSS assessment is based on the TIMSS curriculum model, which considers how 
educational opportunities are provided to pupils and the factors that influence how pupils 
use these opportunities. The model captures the mathematics and science that most 
students are expected to learn and how an educational system should be organised to 
facilitate this learning. The model has 3 domains: 


1. The national, social and educational context, which informs the creation of the 
intended curriculum 


2. The school, teacher and classroom context, which affects the implemented 
curriculum 


3. Student outcomes and characteristics, which reflect the attained curriculum. 


Underpinning the first domain is an encyclopedia documenting education policies and 
curricula in all countries participating in TIMSS°°. The second and third domains form the 
basis of the TIMSS contextual (pupil and teacher) questionnaires and pupil assessment. 


2.1 How was TIMSS administered? 


For the first time, TIMSS 2019 offered participating countries an option to administer the 
assessment in a digital format, eTIMSS. England chose to participate in eTIMSS with 
tests and questionnaires administered on handheld computer tablets using specially 
developed IEA software. In addition to being easier to administer (item development, 
printing, shipping, data entry and scoring were all more efficient), the computerised 
TIMSS tests facilitated assessment of complex areas of the curriculum model that are 
difficult to measure with paper and pencil. 


The eTIMSS assessment was designed to maintain continuity with previous paper-based 
cycles so that countries that chose this assessment option were able to preserve their 
trend measurements. In England a further 1,500 pupils were recruited to sit a paper- 
based bridging study used to link the eTIMSS assessment to the historic TIMSS 
assessment scale. 


2.2 How were the TIMSS scores calculated? 


The main measures of mathematics and science performance in TIMSS are the average 
scores, which are calculated for each participating country based on the scores achieved 
by pupils who took the TIMSS assessments. The full distribution of TIMSS average 
scores is centred at 500, corresponding to the average of the overall achievement 
distribution, with 100 points on the scale corresponding to one standard deviation. The 
scale was established in TIMSS 1995 and linked to the subsequent TIMSS assessment 


38 The TIMSS 2019 Encyclopedia: Education Policy and Curriculum in Mathematics and Science 
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cycles to allow the achievement scores in a given subject and year group to be compared 
over time and across countries. Reference will be made throughout the report to the 
TIMSS centrepoint of 500 and average scores, except with respect to the international 
benchmarks, which use international medians as the average measure. 


Every average score calculated using the TIMSS data is accompanied by a standard 
error (SE) indicating how precisely the sample average can be generalised for the 
population. Standard errors are used to calculate confidence intervals (at the 95% level) 
for all the TIMSS average scores. The lower the standard error, the less uncertainty there 
is due to sampling variations and, therefore, the better the TIMSS sample is as an 
estimate of the whole population’s performance. 


In addition to providing overall scores in mathematics and science, TIMSS enables a 
detailed comparison of pupils’ mathematics and science performance in specific subject 
and cognitive domains (see Table 4 below). Each of the assessment questions is 
categorised according to the area of the curriculum it covers (referred to in TIMSS as 
content domains) and the different cognitive skills it requires (referred to in TIMSS as 
cognitive domains)*°. 


Table 5: Content and cognitive domains in TIMSS 


Domain Year 5 Year 9 
Mathematics content Number; measurement and | Number; algebra; 
domains geometry; data geometry; data and 

probability. 
Science content domains _ | Life science; physical Biology; chemistry; 
science; Earth science. physics; Earth science. 
Cognitive domains in Knowing; reasoning; Knowing; reasoning; 
mathematics and science | applying applying 


The TIMSS performance scales are not constructed to be comparable across subjects 
and year groups as they measure different competences. However, because the scores 
in each subject and each year group are based on parallel scales and are nationally 
representative, it is possible to compare the relative position of pupils in different 
countries at any point in time. If the same cohort of pupils is studied in a subsequent 
cycle of TIMSS, it is possible to gain insights how well that same cohort of pupils has 
performed over time, relative to the TIMSS international centrepoint in each study*". 


40 See the TIMSS 2019 Frameworks: Mullis, |. V. S., & Martin, M. O. (Eds.). (2017). TIMSS 2019 
Assessment Frameworks. Retrieved from Boston College, TIMSS & PIRLS International Study Center 


website: http://timssandpirls.bc.edu/timss2019/frameworks/ 
41 Pupils in the sample assessed in 2015, when they were in year 5, will not necessarily be the same as 
pupils in the sample of year 9 pupils assessed in 2019. 
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2.3 The TIMSS international benchmarks 


In each TIMSS cycle the distribution of pupil scores is described using a set of 
international benchmarks that reflect different levels of pupil achievement. There are 4 
benchmarks each in mathematics and science, and these are designed to be comparable 
over time. A score of 625 indicates that a pupil has reached an advanced level, a score 
of 550 indicates a high level, a score of 475 indicates an intermediate level and a score 
of 400 indicates a low level of application. Tables 5 and 6 below present the main 
statements describing the application of knowledge and understanding required for pupils 
to achieve these benchmarks: full descriptions are given in Appendix C. 


Table 6: International benchmarks for TIMSS mathematics achievement at years 5 
and 9 (scores required to reach each benchmark) 


Year 5 international benchmarks 


Year 9 international benchmarks 


Advanced (625): Students can apply their 
understanding and knowledge in a variety 
of relatively complex situations and explain 
their reasoning. 


Advanced (625): Students can apply and 
reason in a variety of problem situations, 
solve linear equations and make 
generalisations. 


High (550): Students apply conceptual 
understanding to solve problems. 


High (550): Students can apply their 
understanding and knowledge in a variety 
of relatively complex situations. 


Intermediate (475): Students can apply 
basic mathematical knowledge in simple 
situations. 


Intermediate (475): Students can apply 
basic mathematical knowledge in a 
variety of situations. 


Low (400): Students have some basic 
mathematical knowledge. 


Low (400): Students have some 
knowledge of whole numbers and basic 
graphs. 


Source: TIMSS 2019. 


Table 7: International benchmarks for TIMSS science achievement at years 5 and 9 
(scores required to reach each benchmark) 


Year 5 international benchmarks 


Year 9 international benchmarks 


Advanced (625): Students communicate 
their understanding of life, physical and 
Earth sciences and 

demonstrate some knowledge of the 
process of scientific enquiry. 


Advanced (625): Students communicate 
understanding of concepts related to 
biology, chemistry, physics and Earth 
science in a variety of contexts. 


High (550): Students communicate and 
apply knowledge of the life, physical and 
Earth sciences. 


Intermediate (475): Students show 
knowledge and understanding of some 
aspects of life, physical and Earth 
sciences. 


High (550): Students apply understanding 
of concepts from biology, chemistry, 
physics and Earth science. 


Intermediate (475): Students show and 
apply some knowledge of biology, 
chemistry and the physical sciences. 
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Year 5 international benchmarks Year 9 international benchmarks 
Low (400): Students show limited Low (400): Students show limited 
understanding of scientific concepts and understanding of scientific principles and 
limited Knowledge of foundational science | concepts and limited knowledge of 
facts. scientific facts. 


Source: TIMSS 2019. 


2.4 Educational experience of the TIMSS cohorts 


The year 5 and 9 pupils who participated in the study have experienced different 
curriculum and assessment arrangements during their schooling and this may have 
influenced their achievement and attitudes to learning. 


The year 5 pupil cohort for TIMSS 2019 


The year 5 pupils who completed TIMSS 2019 were typically born in 2008 or 2009, and 
entered full-time education from September 2013. They were the first year group to be 
taught entirely according to the National Curriculum in England: framework for key stages 
1 to 4 (DfE, 2013), as they started year 1 when this became a statutory requirement in 
September 201447. 


These pupils were assessed in mathematics at the end of key stage 1 in 20164, with 
teachers using statutory tests newly introduced that year and comprising separate 
arithmetic and reasoning papers. These tests were used to inform statutory teacher 
assessments for pupils in mathematics that were used for formal accountability 
measures. Teacher assessments for science were reported but not used for formal 
accountability. All teacher assessment used the Interim Teacher Assessment 
Frameworks (Standards and Testing Agency, 2015), including interim pre-key stage 
standards for pupils working below the standard of statutory testing arrangements. 


The year 9 pupil cohort for TIMSS 2019 


The year 9 pupils who completed TIMSS 2019 were typically born in 2004 or 2005, 
entering full-time education from September 2009. They were taught according to the 
previous national curriculum (DfEE, 1999) up until September 2013. For the academic 
year 2013/14, when these pupils were in year 4, the government disapplied the previous 
national curriculum (DfEE, 1999) to aid transition to the latest national curriculum (DfE, 
2013). Schools were able to choose whether to use the previous curriculum, or to start 
using the new one a year earlier. From September 2014, the revised programmes of 
study in mathematics and science in the latest national curriculum (DfE, 2013) became 
statutory . The year 9 pupils were therefore all taught using the latest national curriculum 
from September 2014, when they were in year 5. 


42 Differences might occur if pupils were taught in academies using their discretion not to teach the national 
curriculum, or for pupils in independent schools. 


43 See https://www.gov.uk/quidance/2016-key-stage-1-assessment-and-reporting-arrangements- 
ara/section-8-teacher-assessment 
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The pupils were the first to be assessed against the latest national curriculum at the end 
of key stage 2 in 2016.4 Statutory mathematics tests (SATs) comprised 1 arithmetic 
paper and 2 reasoning papers. Science testing for all pupils was not statutory in 2016. 
Instead, biennial science sampling tests were administered in selected schools by 
external administrators — in these selected schools, participation was 

statutory. Pupils also received teacher assessments of achievement in mathematics 
and science. 


In addition, this cohort of pupils was in year 5 at the time of the previous TIMSS 2015 
assessment. This enables some comparison of this cohort’s progress over time using 
representative samples from each cycle of TIMSS assessments. 


The TIMSS Encyclopedia* chapter for England provides more detail about the education 
context in England at the time of the TIMSS tests. 


2.5 To what extent were the TIMSS topics taught in 
England prior to the 2019 assessments? 


TIMSS assesses year 5 and 9 pupils in a number of mathematics and science topics. 
The IEA reports the extent to which these topics are intended to be taught to pupils in 
these year groups so that the level of curriculum match can be established. Full 
information on the curriculum match for other countries can be found in the TIMSS 
International Report 2019 and the TIMSS encyclopedia. 


Overall, in England, the TIMSS 2019 assessments are well matched to the content of the 
national curriculum (DfE, 2013), both in mathematics and science. This revised national 
curriculum was made statutory for local authority maintained schools in England in 
September 2014. These pupils who undertook the TIMSS assessments in England had 
been taught this curriculum for 5 academic years (years 1 to 5 for the year 5 pupils and 
years 5 to 9 for the year 9 pupils). Pupils in non-local authority schools such as 
academies during this period were required to be taught a broad and balanced 
curriculum that includes English, mathematics and science. 


A high level of curriculum match is not necessarily associated with high levels of 
performance. For example, Singapore was the highest-achieving country for science in 
year 9, but it had taught only 14 of the 26 TIMSS topics by the time its pupils took their 
TIMSS assessments. 


Year 5 


The national curriculum in England is arranged into 4 key-stage sections. For schools 
following the national curriculum, there is a higher level of confidence in the topics 
covered by the end of each key stage period. The year 5 TIMSS pupils in England were 
only part way through the relevant key stage, so it is not known which TIMSS topics they 
had covered by the time they took the assessments. The national curriculum provides 
guidance on splitting work up over the key stage period and this was used to assess how 
many topics were likely to have been covered by the year 5 pupils participating in TIMSS. 


44 See https://www.gov.uk/guidance/2016-key-stage-2-assessment-and-reporting-arrangements-ara 
45 The TIMSS 2019 Encyclopedia: Education Policy and Curriculum in Mathematics and Science 
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In mathematics, 14 out of 17 topics included in the TIMSS assessments were intended to 
be taught by the end of year 5, with only 3 measurement and geometry topics not 
expected to be covered (solving problems involving length including measuring and 
estimating; solving problems involving mass, volume and time; and finding and 
estimating perimeter, area and volume). 


In science, 21 of the 26 topics were intended to be taught to year 5 pupils. Four Earth 
science topics (Earth’s resources used in everyday life; changes in Earth’s surface over 
time; weather and climate; Earth’s motion and related patterns observed on Earth) and 1 
physical science topic (heat transfer) included in the TIMSS assessments were not part 
of the national curriculum for pupils up to this age. 


Year 9 


The national curriculum provides guidance on work to be covered by year 9 pupils in 
English schools but, as the pupils were only part way through the academic year, it is not 
known which TIMSS topics they had covered by the time they took the assessments. 


In mathematics, all 22 of the TIMSS topics were intended to be taught by the end of year 
9 in England. 


In science, 25 of the 26 topics were intended to be taught by the end of year 9, with only 
1 chemistry topic (the role of electrons in chemical bonds) included in the assessments 
that does not form part of the national curriculum for pupils up to this age. 


Sample TIMSS items 


The sample test items cover a range of questions used to test pupils at the high and low 

international benchmarks for mathematics and science in both years 5 and 9. The format 
of the items is similar to national assessment items. A selection of the questions used in 

TIMSS 2019 is published in the IEA’s TIMSS International Report 2019%°. 


46 Available at https://timssandpirls.bc.edu/timss2019/ 
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Chapter 3. Overall performance in mathematics 


This chapter summarises the findings from TIMSS 2019 on mathematics performance for 
year 5 and year 9 pupils in England. It covers the changes in average performance over 
time and changes in the percentage of pupils reaching each of the international 
benchmarks for achievement in mathematics*’. The comparator countries referred to in 
this chapter are listed in section 1.548. 


3.1 Main findings 


In 2019, the performance of pupils in both year 5 and year 9 in mathematics in 
England was significantly above the TIMSS centrepoint. 

The trend in England’s year 5 mathematics score is one of improvement over time, 
from significantly below the TIMSS centrepoint in 1995 to significantly above in 
2019. The increase in England’s average score (10 scale points) between 2015 
and 2019 meant year 5 pupils’ performance in 2019 was significantly above all 
previous TIMSS cycles. 

England’s performance in year 9 mathematics has seen significant improvement 
over the last 24 years, most notably between 2003 and 2007, and has been 
broadly stable since 2007. 

For year 5, 7 countries performed significantly above England, 1 at a similar level, 
and 49 significantly below. The same 7 countries also performed significantly 
above England in 2015: the 5 East Asian countries (Chinese Taipei, Hong Kong, 
Japan, the Republic of Korea and Singapore), Northern Ireland and Russia. 

For year 9, 6 countries performed significantly above England, 7 at a similar level, 
and 25 significantly below. The same 6 countries also performed significantly 
above England in 2015: the 5 East Asian countries and Russia. 

A larger share of year 5 and 9 pupils reached each of the international 
benchmarks in England compared with the international median across all 
participating countries. 

In 2019, the percentages of year 5 pupils reaching the advanced and the high 
benchmark or above were significantly higher than in all previous TIMSS cycles, 
except 2011. 

Between 1995 and 2019, there was a significant improvement in the percentage of 
year 9 pupils in England the intermediate, advanced and high international 
benchmarks, although the total percentage of pupils who reached the low 
benchmark or above did not increase significantly. Although 3% more year 9 
pupils performed below the low benchmark in 2019 than in 2015, this was not 
significant. 

In 2019, England had a relatively large difference between its highest- and lowest- 
performing year 5 pupils (a range of 282 scale points) and year 9 pupils (a range 
of 297 scale points). At year 5, most of the highest-performing and European 


47 See Appendix D for a guide to interpreting the benchmark charts in this chapter. 
48 Canada, Ireland, Germany, the Netherlands, Poland and Spain did not participate in the year 9 study. 
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comparator countries had smaller ranges. However, at year 9, while still larger 
than the range of scores in all European comparator countries, England’s range 
was smaller than in some of the highest-performing countries: Chinese Taipei, 
Hong Kong and the Republic of Korea. 

¢ TIMSS allows for a comparison of a cohort’s performance over 2 cycles as year 9 
pupils in 2019 were in year 5 in 2015. Relative to the TIMSS centrepoint, this 
cohort of pupils performed better in year 5 than in year 9. A similar decline in 
performance relative to the TIMSS centrepoint was also reported in some of the 
comparator countries (for example, Ireland, Russia and the United States). 
However, the highest-performing countries generally either maintained their 
positions or secured greater progress over time. 


3.2 What does TIMSS tell us about England’s performance 
in year 5 mathematics? 


3.2.1 How has England’s performance in mathematics changed over 
time for year 5 pupils? 


The trend in mathematics performance for year 5 pupils in England is one of 
improvement over time, from significantly below the TIMSS centrepoint*? in 1995 to 
significantly above it in 2019°°. There were significant increases in 2003, 2007 and 2019, 
with performance remaining broadly stable between 2007 and 2015. The 10 scale-point 
increase in average score in 2019 was the first significant increase since 2007 and meant 
pupils’ performance was significantly above all previous TIMSS cycles. The 2019 
average mathematics score for England (556) was significantly above the TIMSS 
centrepoint (500). 


Figure 9 below shows this improvement over time and how this relates to the TIMSS 
centrepoint (500); scores marked with an asterisk were significantly above the previous 
score. In 1995, the TIMSS sample comprised year 4 and 5 pupils. This may have 
affected average achievement levels for that year, and therefore the significance levels of 
the difference in average scores between 1995 and the 2003 cycle. 


49 ‘The TIMSS achievement scales were established in TIMSS 1995 based on the achievement distribution 
across all participating countries, treating each country equally. At each grade level, the scale centerpoint 
of 500 was set to correspond to the 1995 mean of the overall achievement distribution, and 100 points on 
the scale was set to correspond to the standard deviation. Achievement data from subsequent TIMSS 
assessment cycles were linked to these scales so that increases or decreases in average achievement 
may be monitored across assessments. TIMSS uses the scale centerpoint as a point of reference that 
remains constant from assessment to assessment’ (see https://nces.ed.gov/timss/faq.asp). 

50 Significance levels will depend on the averages but also on the standard deviations. Both averages and 
standard deviations are used to calculate a T-statistic, which is then compared to the critical values in t- 
tables. 
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Figure 9: Trend in average year 5 mathematics score (England) 
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Source: TIMSS 2019. 
Note 1: The 1999 cycle of TIMSS included only year 9 pupils, represented by the dashed 
line. 
Note 2: The 1995 score is an average across the performance of year 4 and year 5 
pupils as the 1995 cycle assessed pupils across both year groups. 
Note 3: Response rates for TIMSS in England were relatively low in 1995 and 2003. 
Note 4: Mathematics scores that represent a significant increase on the previous TIMSS 
cycle are marked with an asterisk (*). 


Figure 10 below shows the percentage of year 5 pupils in England meeting each of the 
international TIMSS benchmarks®' in mathematics since 1995. The chart is cumulative 
so that, reading left to right, it presents the percentage of pupils who reached all of the 
benchmarks from advanced to low or above. For example, in 2019 in England 21% of 
pupils reached the advanced benchmark, 53% the high benchmark or above, 83% the 
intermediate benchmark or above and 96% the low benchmark or above. The remaining 
4% (not shown in Figure 10) did not reach the low benchmark. 


Between 1995 and 2019 there was significant improvement in the percentage of year 5 
pupils in England reaching each of the international benchmarks. In 2019, compared to 
all previous TIMSS cycles, except 2011, significant percentages of pupils reached the 
advanced benchmark and high benchmark or above. The percentages of pupils reaching 
the intermediate and low benchmarks or above were also significantly above all cycles 
except 2015. Since 1995, the share of pupils in England reaching the high benchmark or 


51 See Section 2.2 and Appendix C for descriptions of the international benchmarks. See Appendix D for a 
guide to interpreting the benchmark charts. 
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above more than doubled (from 24% to 53%), while the percentage reaching the 
advanced benchmark trebled (from 7% to 21%). Overall, the percentage of pupils 
reaching the low benchmark or above in 2019 was the same as in 2015 (96%). 


Figure 10: Trend in the percentage of year 5 pupils reaching each of the TIMSS 
international benchmarks in mathematics (England) 
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Source: TIMSS 2019. 
Note 1: The 1999 cycle of TIMSS included only year 9 pupils. 
Note 2: The 1995 score is an average across the performance of year 4 and year 5 
pupils as the 1995 cycle assessed pupils across both year groups. 
Note 3: Response rates for TIMSS in England were relatively low in 1995 and 2003. 


3.2.2 How did year 5 pupils in England perform in mathematics relative 
to their peers in all other TIMSS countries? 


Fifty-eight countries participated in the TIMSS 2019 year 5 mathematics assessments. 
Full international analyses of their performance can be found in the TIMSS International 
Report 2019. 


In 2019, 7 countries performed significantly above England, Ireland performed at a 
similar level and the remaining 49 countries performed significantly below England. The 7 
countries that performed significantly above England in 2019 were the same as in 2015: 
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the 5 East Asian countries (Chinese Taipei, Hong Kong, Japan, the Republic of Korea 
and Singapore), Northern Ireland and Russia. 


The composition of the group performing at a similar level to England in 2019 has 
changed since 2015. Ireland is now the only country in this category, with a non- 
significant 1 scale-point increase in average achievement. All of the other countries in 
this category in 2015 had lower average achievement in 2019, in contrast to England’s 
significantly higher average achievement. The decreases for Norway and the United 
States were not significant, but those for the remaining countries were. 


Between 2015 and 2019, the gap between the performance of pupils in England and the 
performance of their peers in the group of countries performing significantly above 
England has narrowed. In 2015, there was an 18-point gap between Russia, the lowest- 
achieving country from this group, and England (564 compared with 546). In 2019, there 
was a 10-point difference by the same measure between Northern Ireland and England 
(566 compared with 556). 


Tables 7, 8 and 9 below show how England’s year 5 pupils performed in 2015 and 2019 
relative to those in a selection of other countries by average score. England’s average 
score was 546 in 2015 and 556 in 2019. 


Table 8: Year 5 mathematics: all countries performing significantly above England 
in 2015 and 2019 (average scores) 


2015 2019 
Singapore (618) Singapore (625) 
Hong Kong (615) Hong Kong (602) 
Republic of Korea (608) Republic of Korea (600) 
Chinese Taipei (597) Chinese Taipei (599) 
Japan (593) Japan (593) 
Northern Ireland (570) Russia (567) 
Russia (564) Northern Ireland (566) 


Table 9: Year 5 mathematics: all countries performing at a similar level to England 
in 2015 and 2019 (average scores) 


2015 2019 
Norway (549) England (556) 
Ireland (547) Ireland (548) 
England (546) 
Portugal (541) 
United States (539) 
Denmark (539) 
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Table 10: Year 5 mathematics: comparator group countries performing 
significantly below England in 2015 and 2019, including countries that performed 
similarly to England in 2015 (average scores) 


2015 


2019 


Lithuania (536) 
Finland (535) 
Poland (535) 
Netherlands (530) 
Germany (522) 
Sweden (519) 
Australia (517) 
Canada (511) 
Italy (507) 

Spain (505) 

New Zealand (491) 
France (488) 

and 24 others 


Norway (543) 
Lithuania (542) 
Netherlands (538) 
United States (535) 
Belgium (Flemish) (532) 
Finland (532) 
Portugal (525) 
Denmark (525) 
Sweden (521) 
Germany (521) 
Poland (520) 
Australia (516) 
Italy (515) 

Canada (512) 
Spain (502) 

New Zealand (487) 
France (485) 

and 32 others 


Source: TIMSS 2019. 


3.2.3 How did year 5 pupils in England perform in mathematics relative 
to their peers in the comparator countries? 


TIMSS scores 


In this section, comparisons are drawn between the performance of England’s year 5 
pupils and pupils from the 3 comparator groups: highest-performing, English-speaking 
and European (see Section 1.5). Trends are shown for countries with at least 3 cycles of 
assessment data since 2003. In some cases, countries have been selected so that no 
more than 5 comparator countries are shown in each figure for clarity of presentation. 
Where this is the case, selection has primarily focused on the length of trend data 
available to draw comparisons with the performance of pupils in England over time. 


The term ‘highest-performing’ is used to describe the group that comprises the 5 East 
Asian countries and Russia. All of the countries from this group in Figure 11 below, 
alongside England, have seen significant improvement in year 5 pupils’ mathematics 
performance between 2003 and 2019 (although Hong Kong had a significant decrease in 
performance in 2019). In contrast to England, all of these countries saw significant 
improvements in their performance in 2015. However, in 2019, England was the only one 
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of these countries that had a significantly higher average score compared with 2015. 
Despite England’s relative improvement, year 5 pupils from all these countries performed 
significantly above pupils from England in 2019. 


Figure 11: Trends in year 5 mathematics performance between 2003 and 2019 for 
England and highest-performing comparator countries 
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Source: TIMSS 2019. 


Note 1: Comparator countries are only included if they have at least 3 cycles of data 
since 2003. 


Three of the English-speaking comparator countries have time series data from 2003: 
Australia, New Zealand and the United States. Like England, Australia and the United 
States have seen significant improvements in performance between 2003 and 2019, 
while the trend in performance for New Zealand is one of stability over time (see Figure 
12 below). Ireland and Northern Ireland have a shorter time series of data, showing a 
significant improvement between 2011 and 2019 for Ireland and stability for Northern 
Ireland. Compared to all countries in this group, only pupils in England performed 
significantly higher in 2019 than in 2015. Pupils from Northern Ireland performed 
significantly above their peers in England in 2019, while those in Ireland performed at a 
similar level. Pupils in England performed significantly above the remaining countries 
(including Canada as an English-speaking country not shown in Figure 12). 
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Figure 12: Trends in year 5 mathematics performance between 2003 and 2019 for 
England and selected English-speaking comparator countries 
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Source: TIMSS 2019. 


Note 1: Comparator countries are only included if they have at least 3 cycles of data 
since 2003. 


Like England, both Italy and Lithuania have seen significant improvements in 
performance between 2003 and 2019 (see Figure 13 below), while the Netherlands’ 
performance was stable over time. For Germany and Sweden, data were available from 
the 2007 cycle but not 2003. Sweden’s performance improved significantly between 2007 
and 2019, while Germany’s remained stable. Like England, Italy’s and the Netherlands’ 
performance was significantly higher in 2019 than in 2015. England’s pupils performed 
significantly above their peers from all the countries in the European comparator group in 
2019. 
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Figure 13: Trends in year 5 mathematics performance between 2003 and 2019 for 
England and selected European comparator countries 
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Note 1: Comparator countries are only included if they have at least 3 cycles of data 
since 2003. 


Figure 14 below shows the range of year 5 mathematics scores in England against the 
countries from the 3 comparator groups from the 5" percentile (low-performing pupils) to 
the 95'" percentile (high-performing pupils) on the distribution of scores in each country. 
The range is not calculated using the difference between the maximum and minimum 
scores because of the potential distortion due to outliers. The dark section in the centre 
of each bar represents the average score for year 5 mathematics and the 95% 
confidence interval around it. 


Year 5 pupils in England at the lower end of the distribution (the 5" percentile) achieved 
an average score of 407 in 2015 and 411 in 2019, a small increase of 4 scale points. At 
the top end of the distribution (the 95" percentile), pupils achieved an average score of 
682 in 2015 and 693 in 2019, a larger increase of 11 scale points. In combination, these 
average score changes have increased the achievement gap by 7 scale points from 275 
in 2015 to 282 in 2019. Referring back to Figure 10, it can be seen that the increase in 
the range of scores in England between 2015 and 2019 was driven by the change at the 
top end of the distribution with more pupils achieving the advanced benchmark for year 5 
mathematics. 


Most of the highest-performing countries as well as those from other comparator groups 
had a smaller range than England’s. However, like England, 3 of the 6 other English- 
speaking countries also had relatively large ranges: Australia, Northern Ireland and the 
United States. Data on all other participating countries are available in the TIMSS 
International Report 2019. 
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Figure 14: Range of year 5 mathematics achievement between the lowest- and 
highest-performing pupils across comparator countries 
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Source: TIMSS 2019. 
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TIMSS international benchmarks 


As shown in Figure 15 below, fewer pupils in England reached the advanced benchmark 
compared to their peers in any of the high-performing countries, except in Russia, while 
this was the case in comparison with all these countries for the high benchmark or above. 
For example, more than twice the percentage of year 5 pupils in the highest-performing 
country, Singapore, reached the advanced benchmark as those in England (54% 
compared with 21%). Furthermore, 84% of pupils in Singapore reached the high 
benchmark or above compared with 53% in England (although both countries’ 
percentages increased by the same 4 points compared with 2015). In addition, 96% of 
pupils in Singapore reached the intermediate benchmark or above, compared with 83% 
of pupils in England. 


Nevertheless, compared with the international median across all participating countries, a 
larger share of pupils in England reached each benchmark°*?, with 3 times as many pupils 
in England (21% compared with 7%) reaching the advanced benchmark. 


More of England’s pupils also reached each benchmark compared to their peers in most 
other English-speaking countries. The exceptions were Ireland, where a lower 
percentage of pupils reached the advanced benchmark, but similar percentages reached 
the remaining 3 benchmarks or above. Also, in Northern Ireland, more pupils reached all 
benchmarks than in England, except the low benchmark or above. 


More pupils in England reached the advanced benchmark and high benchmark or above 
compared to their peers in all the European comparator countries. This was the same for 
England’s pupils reaching the intermediate benchmark or above, except in comparison 
with pupils in the Netherlands. More pupils in England reached the low benchmark or 
above compared to their peers in this group, except in comparison with pupils in 
Germany, Lithuania, the Netherlands and Norway. 


Data on all other participating countries are available in the TIMSS International Report 
2019. 


52 The IEA calculates international medians rather than international averages for this data set. 
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Figure 15: Percentage of year 5 pupils reaching the international benchmarks in 
year 5 mathematics (England and comparator countries) 
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Source: TIMSS 2019. 
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3.3 What does TIMSS tell us about England’s performance 
in year 9 mathematics? 


3.3.1 How has England’s performance in mathematics changed over 
time for year 9 pupils? 


England’s performance in year 9 mathematics has seen significant improvement over the 
last 24 years, most notably between 2003 and 2007, with performance remaining broadly 
stable since 2007 (see Figure 16 below). In 2019, year 9 pupils in England performed 
above the TIMSS centrepoint, as they have since 2007. The 2019 average mathematics 
score for England was 515, 3 scale points lower than 2015, but this decrease was not 
significant. In 1995, the TIMSS sample comprised year 8 and 9 pupils, and this may have 
affected average achievement levels for that year and corresponding comparisons made. 


Figure 16: Trend in average year 9 mathematics score (England) 
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Source: TIMSS 2019. 
Note 1: The 1995 score is an average across the performance of year 8 and year 9 
pupils as the 1995 cycle assessed pupils across both year groups. 
Note 2: Response rates for TIMSS in England were relatively low in 1995, 1999 and 
2003. 
Note 3: Scores that represent a significant increase on the previous TIMSS cycle are 
marked with an asterisk. 
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Figure 17 below shows the percentage of year 9 pupils in England meeting each of the 
international TIMSS benchmarks®? in mathematics since 1995. The chart is cumulative 
so that, reading left to right, it presents the percentage of pupils who reached all of the 
benchmarks from the advanced benchmark to the low benchmark or above. For 
example, in 2019 in England 11% of pupils reached the advanced benchmark, 35% the 
high benchmark or above, 69% the intermediate benchmark or above and 90% the low 
benchmark or above. The remaining 10% did not reach the low benchmark. 


Between 1995 and 2019, there has been a significant improvement in the percentage of 
year 9 pupils in England reaching all the international benchmarks, except for the low 
benchmark. The percentage of pupils reaching the advanced benchmark between 1995 
and 2019 has almost doubled (from 6% to 11%). In 2019, the percentages of pupils 
reaching the intermediate or above, the high benchmark or above and the advanced 
benchmark have remained similar to 2015. There was a decrease of 3 percentage points 
for pupils reaching the low benchmark or above in 2019, compared with an increase of 5 
points between 2011 and 2015. However, neither of these was significant. 


Figure 17: Trend in the percentage of year 9 pupils reaching each of the TIMSS 
international benchmarks in mathematics (England) 
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Source: TIMSS 2019. 


53 See Section 2.2 and Appendix C for descriptions of the international benchmarks. See Appendix D for a 
guide to interpreting the benchmark charts. 
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3.3.2 How did year 9 pupils in England perform in mathematics relative 
to their peers in all other TIMSS countries? 


Thirty-nine countries participated in the TIMSS 2019 year 9 mathematics assessments. 
Full international analyses of their performance can be found in the T/MSS International 
Report 2019. 


Year 9 pupils in 6 countries performed significantly above England, in 7 countries 
performed at a similar level, and in 25 countries performed significantly below. 


The 6 countries that performed significantly above England in 2019 were the same as in 
2015: the 5 East Asian countries and Russia. These countries also comprised the group 
that performed significantly above England in the year 5 findings, with the exception of 
Northern Ireland, which did not participate in the year 9 assessments in 2019. 


The countries with similar average achievement to England in 2019 mirror those in 2015 
with the exception of Norway, which performed significantly below England in 2019, and 
Australia which increased its score from 2015. Canada and Slovenia did not participate in 
2019, although Finland did, having not participated in 2015. 


Tables 10, 11 and 12 below show how England’s year 9 pupils performed, in 2015 and 
2019, relative to those in a selection of other countries by average score. England’s 
average score in 2015 was 518 and in 2019 was 515. 


Table 11: Year 9 mathematics: all countries performing significantly above England 
in 2015 and 2019 (average scores) 


2015 


2019 


Singapore (621) 
Republic of Korea (606) 
Chinese Taipei (599) 
Hong Kong (594) 
Japan (586) 

Russia (538) 


Singapore (616) 

Chinese Taipei (612) 

The Republic of Korea (607) 
Japan (594) 

Hong Kong (578) 

Russia (543) 
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Table 12: Year 9 mathematics: all countries performing at a similar level to England 
in 2015 and 2019 (average scores) 


2015 


2019 


Canada (527)** 
Ireland (523) 
England (518) 
United States (518) 
Slovenia (516)** 
Hungary (514) 
Norway (512) 
Lithuania (511) 
Israel (511) 


Ireland (524) 
Lithuania (520) 
Israel (519) 
Australia (517) 
Hungary (517) 
England (515) 
United States (515) 
Finland (509)* 


Table 13: Year 9 mathematics: comparator countries performing significantly 


below England in 2015 and 2019 (average scores) 


2015 


2019 


Australia (505) 
Sweden (501) 

Italy (494) 

New Zealand (493) 
and 20 others 


Norway (503) 
Sweden (503) 

Italy (497) 

France (483)* 

New Zealand (482) 


and 20 others 


Source: TIMSS 2019. 
Note 1: Average achievement is shown in parentheses. 
Note 2. Asterisk (*) denotes countries that did not participate in the 2015 TIMSS year 9 
mathematics assessments. 
Note 3. Double asterisk (**) denotes countries that did not participate in 2019 TIMSS year 
9 assessments. 


3.3.3 How did year 9 pupils in England perform in mathematics relative 
to their peers in the comparator countries? 


TIMSS scores 


In this section, comparisons are drawn between the performance of England’s year 9 
pupils and pupils from the 3 comparator groups: highest-performing, English-speaking 
and European (see Section 1.4). Trends are shown for countries with at least 3 cycles of 
assessment data since 2003. In some cases, countries have been selected so that no 
more than 5 comparator countries are shown in each figure for clarity of presentation. 
Where this is the case, selection has primarily focused on the length of trend data 
available to draw comparisons with the performance of pupils in England over time. 
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The performance of pupils in each of the highest-performing countries (the 5 East Asian 
countries and Russia) was significantly above that of England’s pupils in 2019. Apart 
from Hong Kong and Singapore, where performance has remained stable between 2003 
and 2019, all of the highest-performing countries have, like England, seen a significant 
improvement in year 9 pupils’ mathematics performance over the same time period (see 
Figure 18 below). While, like England, the trend for most of these countries has been one 
of stability or significant improvement between cycles, Chinese Taipei, Hong Kong and 
Singapore have seen significant decreases in average scores in 1 or more cycles. 


Figure 18: Trends in year 9 mathematics performance between 2003 and 2019 for 
England and highest-performing countries 
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Source: TIMSS 2019. 


Note 1: Comparator countries are only included in the chart if they have at least 3 cycles 
of data since 2003. 


Of the English-speaking countries, Australia and England are the only countries to show 
significant improvement between 2003 and 2019 (see Figure 19 below). New Zealand’s 
trend had been of stability over time until 2019, when its performance was significantly 
below that recorded in 2015. The other countries’ trends have been stable overall but 
with each recording a significant increase in average score in 1 cycle: Australia in 2019, 
England in 2007 and the United States in 2015. In 2019, year 9 pupils in England 
performed similarly to their peers in Australia, Ireland and the United States and 
significantly above pupils in New Zealand. 
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Figure 19: Trends in year 9 mathematics performance between 2003 and 2019 for 
England and English-speaking comparator countries 
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Source: TIMSS 2019. 


Note 1: Comparator countries are only included if they have at least 3 cycles of data 
since 2003. 

Note 2: New Zealand did not participate in year 9 TIMSS in 2007, represented by the 
dashed line. 


Like England, among the European comparator countries both Italy and Lithuania have 
seen significant improvement in year 9 pupils’ mathematics performance between 2003 
and 2019. However, this was not the case for Sweden, where performance decreased 
significantly between 2003 and 2011, before improving to 2003 levels in 2015, and 
remaining stable in 2019 (see Figure 20 below). Lithuania has recorded significant 
improvements in its performance in the last 2 cycles. Including countries from the wider 
European comparator group, in 2019, England’s pupils’ performance was similar to their 
peers in Finland and Lithuania and significantly above that of their peers in the remaining 
countries. 
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Figure 20: Trends in year 9 mathematics performance between 2003 and 2019 for 
England and European comparator countries 
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Note 1: Comparator countries are only included if they have at least 3 cycles of data 
since 2003. 


Figure 21 below shows the range of year 9 mathematics scores in England against the 
countries from the 3 comparator groups from the 5" percentile (low-performing pupils) to 
the 95th percentile (high-performing pupils) on the distribution of scores in each country. 


Year 9 pupils in England at the lower end of the distribution (the 5" percentile) achieved 
an average score of 389 in 2015 and 363 in 2019, a decrease of 26 scale points. By 
contrast, at the top end of the distribution (the 95" percentile), pupils achieved an 
average score of 649 in 2015 and 660 in 2019, an increase of 11 scale points. This 
decrease in performance for lower-achieving pupils, combined with an increase for the 
higher-achieving pupils, meant the achievement gap between the higher and lower- 
achieving pupils was greater in 2019 (297) than 2015 (260) by 37 scale points. 


Referring back to Figure 17, it can be seen that this increase in the range of scores in 
England between 2015 and 2019 was primarily driven by a change at the bottom end of 
the distribution, with more pupils failing to achieve the low benchmark for year 9 
mathematics. 


This difference between the performance of the highest- and lowest-scoring pupils in 
England was smaller than in some of the highest-performing countries, such as Chinese 
Taipei, the Republic of Korea, and Hong Kong. Of the English-speaking countries, it was 
also smaller than the range found in the United States, similar to Australia’s and New 
Zealand’s ranges but larger than Ireland’s. England’s difference was also larger than the 
ranges found in each of the European comparator group countries. Data on all other 
participating countries are available in the TIMSS International Report 2019. 
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Figure 21: Range of year 9 mathematics achievement between the lowest and 
highest-performing pupils across comparator countries 
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TIMSS international benchmarks 


As shown in Figure 22 below, fewer year 9 pupils in England reached all 4 benchmarks 
compared to their peers in the highest-performing countries. In Singapore, nearly 5 times 
as many pupils reached the advanced benchmark as did those in England (51% 
compared with 11%), while more than twice as many (79% compared with 35%) reached 
the high benchmark or above. Ninety-two per cent of pupils in Singapore reached the 
intermediate benchmark or above compared with 69% in England. However, compared 
with the international median across all participating countries, a larger share of pupils in 
England reached each benchmark. More than double reached the advanced benchmark 
(11% compared with 5%), while the percentage who reached the low benchmark or 
above was just above the international median (90% in England compared with 87%)*4. 


Looking at English-speaking countries, performance against the benchmarks relative to 
England was mixed. The percentage of year 9 pupils in England reaching the advanced 
benchmark was higher than Ireland (11% compared with 7%), but the percentages of 
pupils in England reaching the remaining benchmarks were below the percentages for 
Ireland. The performance of England’s pupils at each benchmark was the same or similar 
to their peers in Australia, while more of England’s pupils reached each benchmark 
compared to their peers in New Zealand. Compared to the United States, England’s 
performance was 3 percentage points lower at both the advanced benchmark and high 
benchmark or above, but it was 3 percentage points higher against the intermediate and 
low benchmarks or above. 


More pupils in England reached the advanced benchmark and high benchmark or above 
compared to their peers in all the European comparator group countries, apart from 
Lithuania (high benchmark or above only). More pupils in England reached the 
intermediate benchmark or above compared to their peers in all these countries, except 
in Finland and Lithuania. However, all European comparators either had a higher 
proportion or the same proportion of pupils reaching the low benchmark or above than 
England, except France. 


Data on all other participating countries are available in the TIMSS International Report 
2019. 


°4 The IEA calculates international medians rather than international averages for this data set. 
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Figure 22: Percentage of year 9 pupils reaching the international benchmarks in 
year 9 mathematics (England and comparator countries) 
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3.4 What does TIMSS tell us about pupil progress in 
mathematics between years 5 and 9? 


As the target year 9 cohort in 2019 was the same cohort of pupils who were in year 5 in 
2015, TIMSS allows for comparison of relative progress achieved by the cohort between 
these years. It should, however, be noted that due to the sampling approach (see section 
1.2), although the year 5 pupils who took the assessments in 2015 were from the same 
cohort, this does not mean they were the same pupils. The assessments taken by year 5 
and year 9 pupils, and the frameworks from which these were taken, were also different. 


As shown in Figure 23, the average score for the year 5 cohort in 2015 in England (546) 
was significantly above the TIMSS centrepoint (500). By the time this cohort reached 
year 9 in 2019, their average score was 15 points above the TIMSS centrepoint (515). 
While still significantly above the centrepoint, this reflects a decrease of 31 scale points in 
terms of relative performance against this centrepoint (546 compared with 515). In 2015, 
this decrease was smaller — 24 scale points (a year 5 scale score of 542 compared with a 
year 9 scale score of 518). 


Such a decrease is also evident for the year 9 cohort in some countries that performed 
similarly to England. For example, the United States saw a decrease of 24 points (539 
compared with 515) and Ireland a decrease of 23 points (547 compared with 524). A 
similar trend occurred in Russia, one of the highest-performing countries (564 compared 
with 543). In the East Asian countries, which comprise the remainder of the highest- 
performing group alongside Russia, progress was mixed. In Japan, the Republic of Korea 
and Singapore, year 5 and year 9 pupils’ performance was similar, while Chinese Taipei 
saw an increase of 15 scale points (597 to 612). This would indicate these countries were 
generally either maintaining or securing greater progress over time. The exception was 
Hong Kong, which saw a decrease larger than England’s (of 37 points, from 615 to 578). 
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Figure 23: A comparison of the mathematics performance of year 5 pupils in 2015 
and year 9 pupils in 2019 (England and other countries from the comparator 
groups) 
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Chapter 4. Overall performance in science 


This chapter summarises the findings from TIMSS 2019 in terms of science performance 
for year 5 and year 9 pupils in England. The chapter covers the changes in average 
performance over time and changes in the percentage of pupils reaching each of the 
international benchmarks in science®>. The comparator countries referred to in this 
chapter are listed in section 1.5°°. 


4.1 Main findings 


The 2019 average science scores for year 5 and 9 pupils in England remained 
significantly above the TIMSS centrepoint. 

Over the 24-year period from 1995 to 2019, the performance in science of year 5 
pupils in England has shown significant improvement. Performance has improved 
significantly since 2011 when there was a significant decline. 

The performance in science of year 5 pupils in England in 2019 remained broadly 
similar to performance in 2015. 

The performance in science of year 9 pupils in England decreased in 2019, 
meaning that it was significantly below all previous TIMSS cycles. 

For year 5, 6 countries performed significantly above England (Chinese Taipei, 
Finland, Japan, the Republic of Korea, Russia and Singapore); this was 3 fewer 
than in 2015. Nine performed at a similar level to England and 42 significantly 
below England. 

For year 9, 9 countries performed significantly above England, 4 more than in 
2015. These included the same 4 East Asian countries as in 2015 (Chinese 
Taipei, Japan, the Republic of Korea and Singapore). Seven performed at a 
similar level to England and 22 significantly below England. 

A larger percentage of year 5 and 9 pupils reached each of the international 
science benchmarks in England compared with the international median across all 
participating countries. 

The percentage of year 5 pupils reaching the low benchmark or above in 2019 
remained relatively stable from 2015. For year 9, the percentage of pupils 
performing below the low benchmark was more than double that of 2015. 


In 2019, the difference between England’s highest- and lowest-performing year 5 
pupils in science was 236 scale points (the median of the comparator group 
countries). At year 9, this range was relatively large (302 scale points). Only New 
Zealand and the United States from the comparator countries had a larger range 
of year 9 science scores. 

TIMSS allows for a comparison of a cohort’s performance over 2 cycles as year 9 
pupils in 2019 were in year 5 in 2015. Relative to the TIMSS centrepoint, this 
cohort of pupils performed better in year 5 science than in year 9 science. A 
similar decline in relative performance was reported in most of the comparator 


°5 See Appendix D for a guide to interpreting the benchmark charts in this chapter. 
°6 Canada, Ireland, Germany, the Netherlands, Poland and Spain did not participate in the year 9 study. 
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countries except Australia, Chinese Taipei, France, Japan, Lithuania and 
Singapore, which reported higher relative performance in year 9 than in year 5. 


4.2 What does TIMSS tell us about England’s performance 
in year 5 science? 


4.2.1 How has England’s performance in science changed over time for 
year 5 pupils? 


Year 5 pupils’ overall performance in science has been consistently and significantly 
above the TIMSS centrepoint (500) for all TIMSS cycles. Performance has improved 
significantly since 2011, following a significant decline between 2003 and 2011. 
England’s average score in 2019 was similar to its scores in 2003 and 2007, and 
significantly above scores in 2011 and 1995 (see Figure 24). It should be noted that in 
1995, the TIMSS sample comprised both year 4 and 5 pupils. This may have affected 
average achievement levels for that year and the level of significance in the subsequent 
cycle, 2003. 


Figure 24: Trend in average year 5 science score (England) 
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Source: TIMSS 2019. 
Note 1: The 1999 cycle of TIMSS included only year 9 pupils, represented by the dashed 
line. 
Note 2: The 1995 score is an average across the performance of year 4 and year 5 
pupils as the 1995 cycle assessed pupils across both year groups. 
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Note 3: Response rates for TIMSS in England were relatively low in 1995, 1999 and 
2003. 

Note 4: Scores that represent a significant increase or decrease from the previous 
TIMSS cycle are marked with an asterisk (*). 


Figure 25 below shows the percentage of year 5 pupils in England meeting each of the 
international TIMSS benchmarks*’ in science since 1995. The chart is cumulative so 
that, read left to right, it presents the percentage of pupils who reached all of the 
benchmarks from advanced to low. For example, in 2019 in England 10% of pupils 
reached the advanced benchmark, 44% the high benchmark or above, 81% the 
intermediate benchmark or above, and 96% the low benchmark or above. The remaining 
4% (not shown in Figure 25) did not reach the low benchmark. 


Overall, the percentages of England’s year 5 pupils reaching the benchmarks are similar 
to those in 2015. Since 2011, there have been non-significant increases in the 
percentage of pupils reaching the high benchmark or above in science, and a significant 
improvement in the percentage of pupils achieving the low and intermediate benchmarks 
or above (3 and 6% percentage points respectively). 


Since 1995, the percentage of pupils reaching the advanced benchmark has shown a 
significant decline, although the percentage reaching the high benchmark or above has 
been more stable. By contrast, the percentages of pupils reaching the intermediate and 
low benchmarks or above show significant improvement since 1995°°. 


57 See Section 2.2 and Appendix C for descriptions of the international benchmarks. See Appendix D for a 
guide to interpreting the benchmark charts. 

°8 Figure calculated using unrounded data. 

59 It should be noted that in 1995, the TIMSS sample comprised both year 4 and 5 pupils, which may have 
affected average achievement levels for that year and the level of trend significance between 1995 and 
2015. 
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Figure 25: Trend in the percentage of year 5 pupils reaching each of the TIMSS 
international benchmarks in science (England) 
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Note 1: The 1999 cycle of TIMSS included only year 9 pupils. 

Note 2: The 1995 score is an average across the performance of year 4 and year 5 
pupils as the 1995 cycle assessed pupils across both year groups. 

Note 3: Response rates for TIMSS in England were relatively low in 1995, 1999 and 
2003. 


4.2.2 How did year 5 pupils in England perform in science relative to 
their peers in all other TIMSS countries? 


Fifty-eight countries participated in TIMSS 2019 year 5 science assessments. Full 
international analyses of their performance can be found in the T/MSS International 
Report 2019. 


In 2019, pupils in 6 countries performed significantly above England; 9 performed at a 
similar level and 42 significantly below. 


In 2019, fewer countries performed significantly above England than in 2015. Those that 
did included the East Asian countries (except Hong Kong), Finland and Russia. Hong 
Kong, Poland and the United States performed similarly to England in 2019 following 
significant decreases in their performance. In 2015, they had all performed significantly 
above England. 


There was some consistency to the list of countries performing similarly to England 
across the 2 cycles as the Czech Republic, Norway and Sweden remained in this group. 
However, both Australia and Lithuania joined this group having performed significantly 
below England in 2015. Hungary, Bulgaria, Croatia and Ireland all performed significantly 
below England in 2019, after performing at a similar level in 2015. 


Tables 13, 14 and 15 below show how England’s year 5 pupils performed, in 2015 and 
2019, relative to those in a selection of other countries by average scale score. England’s 
average scale score was 536 in 2015 and 537 in 2019. 


Table 14: Year 5 science: all countries performing significantly above England in 
2015 and 2019 (average scores) 


2015 


Singapore (590) 
Republic of Korea (589) 
Japan (569) 

Russia (567) 

Hong Kong (557) 
Chinese Taipei (555) 
Finland (554) 

Poland (547) 

United States (546) 


2019 


Singapore (595) 
Republic of Korea (588) 
Russia (567) 

Japan (562) 

Chinese Taipei (558) 
Finland (555) 


Table 15: Year 5 science: all countries performing at a similar level to England in 


2015 and 2019 (average scores) 


2015 


2019 


Slovenia (543)** 
Hungary (542) 
Sweden (540) 
Norway (538) 
Bulgaria (536) 
England (536) 

Czech Republic (534) 
Croatia (533) 

Ireland (529) 


Latvia (542)* 

Norway (539) 

United States (539) 
Lithuania (538) 
Sweden (537) 
England (537) 

Czech Republic (534) 
Australia (533) 

Hong Kong (531) 
Poland (531) 
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Table 16: Year 5 science: comparator countries performing significantly below 
England, including countries that performed similarly to England in 2015 (average 


scores) 
2015 2019 

Germany (528) Hungary (529) 
Lithuania (528) Ireland (528) 
Canada (525) Croatia (524) 
Australia (524) Canada (523) 
Northern Ireland (520) Bulgaria (521) 
Spain (518) Germany (518) 
Netherlands (517) Netherlands (518) 
Italy (516) Northern Ireland (518) 
New Zealand (506) Spain (511) 
France (487) Italy (510) 
and 20 others New Zealand (503) 

France (488) 

and 30 others 


Source: TIMSS 2015 and 2019. 


Note 1. Asterisk (*) denotes countries that did not participate in the 2015 TIMSS year 5 
science assessments. 

Note 2. Double asterisk (**) denotes countries that did not participate in 2019 TIMSS year 
5 science assessments. 


4.2.3 How did year 5 pupils in England perform in science relative to 
their peers in the comparator countries? 


TIMSS scores 


In this section, comparisons are drawn between the performance of England’s year 5 
pupils and pupils from the 3 comparator groups: highest-performing, English-speaking 
and European (see section 1.5). Trends are shown for countries with at least 3 cycles of 
assessment data since 2003. To ensure clarity of presentation, countries have been 
selected so that no more than 5 comparator countries are shown in each figure. Where 
this is the case, selection has primarily focused on the length of trend data available to 
draw comparisons with the performance of pupils in England performance over time. 


In 2019, Hong Kong saw a significant decrease in its pupils’ average achievement in 
science. However, it has been retained as a highest-performing country due to prior 
performance. 


Apart from Hong Kong, all of the highest-performing countries have seen significant 
improvement in year 5 pupils’ science performance between 2003 and 2019 (see Figure 
26 below). This was in contrast to England, where performance over this duration was 
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stable overall with 1 significant decrease in 2011, although performance between 2011 
and 2019 has improved significantly. Hong Kong’s performance has fluctuated, with 2 
cycles showing significant improvement (2007 and 2015) and 2 showing significant 
decreases in average achievement (2011 and 2019). Japan similarly had 2 cycles 
showing significant improvement (2011 and 2015) and a decrease in average 
achievement in 2019. Russia’s performance showed a broadly upward trend, with 2 
significant and 2 non-significant increases, up until 2019 when performance was stable 
compared with 2015. Pupils in all of these countries, except Hong Kong, performed 
significantly above their peers in England in 2019. 


Figure 26: Trends in year 5 science performance between 2003 and 2019 for 
England and highest-performing comparator countries 
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Source: TIMSS 2019. 


Note 1: Comparator countries are only included if they have at least 3 cycles of data 
since 2003. 


Three of the selected English-speaking comparator countries have time series data from 
2003, while 2 (Ireland and Northern Ireland) have these data only from 2011 (see Figure 
27 below). Of these, while the performance of Australia’s year 5 pupils in science 
significantly improved between 2003 and 2019, this was not the case for pupils in 
England, New Zealand or the United States. New Zealand’s 2019 performance was 
significantly below its performance in 2003. Between 2003 and 2011, Australia’s trend 
mirrors England’s, but since then it has seen a significant upward trend in contrast to 
England’s pupils’ stable performance. Northern Ireland’s performance has been stable 
across 3 cycles while Ireland’s improved significantly in 2015. While stable overall, the 
United States’ performance in 2019 was significantly lower than in 2015. 


Across the whole English-speaking comparator group, including Canada, year 5 pupils in 


England performed similarly to those in Australia and the United States in 2019 but 
significantly above Canada, Ireland, New Zealand and Northern Ireland. 
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Figure 27: Trends in year 5 science performance between 2003 and 2019 for 
England and selected English-speaking comparator countries 
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Source: TIMSS 2019. 


Note 1: Comparator countries are only included if they have at least 3 cycles of data 
since 2003. 


Among the European comparator countries shown in Figure 28 below, only in Lithuania 
and Sweden was year 5 pupils’ performance in science in 2019 significantly above the 
performance recorded in 2003 (Lithuania) or 2007 (Sweden). Lithuania showed an 
upward trend in the past 2 cycles, while Germany’s trend has been one of stability until 
its significant decrease in performance in 2019. The Netherlands and Italy have both 
recorded significant increases and decreases since 2003, while Sweden recorded 1 
significant increase. Including data from countries in the wider European comparator 
group reveals that pupils in England performed significantly below their peers in Finland 
in 2019, while pupils in Lithuania, Norway and Sweden performed similarly to England’s 
pupils. Pupils in England performed significantly above those in all the remaining 
European comparator countries. 
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Figure 28: Trends in year 5 science performance between 2003 and 2019 for 
England and selected European comparator countries 
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Note 1: Comparator countries are only included if they have at least 3 cycles of data 
since 2003. 


Figure 29 below shows the range of year 5 science scores from the 5" percentile (lowest- 
performing pupils) to the 95" percentile (highest-performing pupils) on the distribution of 
scores in England and countries from the 3 comparator groups . The range is not 
calculated using the difference between the maximum and minimum scores because of 
the potential distortion due to outliers. The dark section in the centre of each bar 
represents the average score for year 5 science and the 5% confidence interval around 

it. 


Year 5 pupils in England at the lower end of the distribution (the 5" percentile) achieved 
an average score of 417 in 2015 and 413 in 2019, a small decrease of 4 scale points. 
However, at the top end of the distribution (the 95" percentile), pupils achieved similar 
average scores in 2015 (648) and 2019 (649). In combination, these average score 
changes have meant a small increase in the achievement gap, by 5 scale points, from 
231 in 2015 to 236 in 2019. 


In 2019, the difference between England’s highest- and lowest-performing year 5 pupils 
in science was a range of 236 scale points (the median of the comparator group 
countries). While this range was larger than for most of the highest-performing countries, 
it was smaller than Singapore’s (254), the highest-performing country. One of the 
countries in the highest-performing group, Russia, had the smallest range (209) of all 
comparator countries. Amongst English-speaking countries, the United States, Australia, 
lreland and New Zealand had larger ranges than England, while Canada and Northern 
Ireland had similar ranges. England’s range was larger than that found in half of the 
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European comparator countries, the same as Lithuania’s and smaller than the ranges for 
France, Germany, Poland and Sweden. 


Figure 29: Range of year 5 science achievement between the lowest and highest- 
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Source: TIMSS 2019. 


As shown in Figure 30 below, fewer pupils in England reached each benchmark 
compared to their peers in all the highest-performing countries, except in Hong Kong. For 
example, nearly 4 times as many year 5 pupils in the highest performing country, 
Singapore, reached the advanced benchmark compared to those in England (38% 
compared with 10%). However, in Chinese Taipei the difference was much smaller, with 


15% of pupils reaching the advanced benchmark compared with England’s 10%. In 
Singapore, 74% of pupils reached the high benchmark or above compared with 44% in 
England. In addition, 93% of pupils in Singapore reached the intermediate benchmark or 
above, compared with 81% of pupils in England. 


Nevertheless, compared with the international median across all participating countries a 
higher percentage of pupils in England reached each benchmark®, with nearly twice the 
percentage of pupils in England reaching the advanced benchmark (10% compared with 
6%). 


More pupils in England also reached each benchmark than in most of the other English- 
speaking countries. The exceptions were Australia and the United States. In the United 
States, a greater percentage of pupils reached the advanced benchmark and high 
benchmark or above than in England. Australia’s pupils performed similarly to their peers 
in England against each of the benchmarks. 


In relation to the European comparator group countries, fewer pupils in England reached 
the advanced benchmark and high benchmark or above compared to their peers in 
Finland, Lithuania and Sweden. However, more pupils in England reached these 
benchmarks than was the case for their peers in all the remaining countries in this group, 
with the exception of Norway for the high benchmark or above (44% compared with 
46%). Fewer pupils in England reached the intermediate benchmark or above than was 
the case for their peers in Finland and Norway, while the same was the case for the low 
benchmark or above in comparison with pupils in Finland, Lithuania and Norway. 


Data on all other participating countries are available in the TIMSS International Report 
2019. 


80 International medians rather than international averages are calculated for this data set. 
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Figure 30: Percentage of year 5 pupils reaching the international benchmarks in 
science (England and comparator countries) 
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4.3 What does TIMSS tell us about England’s performance 
in year 9 science? 


4.3.1 How has England’s performance in science changed over time for 
year 9 pupils? 


In 2019, England’s year 9 pupils’ overall performance in science remained significantly 
above the TIMSS centrepoint (500), as it has for all previous TIMSS cycles. Performance 
over time between 1995 and 2015 remained broadly stable, with each increase and 
decrease not significant when compared with the previous cycle. However, in 2019, the 
average score for year 9 pupils in England was 517, a significant decrease of 20 scale 
points from 2015 and significantly below average scores in all previous TIMSS cycles. 
Figure 31 below shows this trend over time. 


Figure 31: Trend in average year 9 science score (England) 
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Source: TIMSS 2019. 


Note 1: The 1995 score is an average across the performance of year 8 and year 9 
pupils as the 1995 cycle assessed pupils across both year groups. 

Note 2: Response rates for TIMSS in England were relatively low in 1995, 1999 and 
2003. 

Note 3: Scores that represent a significant increase or decrease from the previous 
TIMSS cycle are marked with an asterisk (*). 
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Figure 32 below shows the percentage of year 9 pupils in England meeting each of the 
international TIMSS benchmarks®' in science since 1995. The chart is cumulative so 
that, read left to right, it presents the percentage of pupils who reached all of the 
benchmarks from advanced to low. For example, in 2019 in England 11% of pupils 
reached the advanced benchmark, 38% the high benchmark or above, 69% the 
intermediate benchmark or above, and 89% the low benchmark or above. The remaining 
11% did not reach the low benchmark. 


In 2019, there was a decrease in the percentage of year 9 pupils in England reaching the 
advanced benchmark, although this was not significant compared with 2015. However, 
the percentages of pupils reaching the high, intermediate and low benchmarks or above 
were all significantly below those recorded in 2015. Prior to 2019, the trend over time in 
each benchmark showed general stability in year 9 pupils’ performance. However, in 
2019, the percentage of pupils not reaching the low benchmark or above more than 
doubled compared with 2015 (from 5% to 11%). 


Figure 32: Trend in the percentage of year 9 pupils reaching each of the TIMSS 
international benchmarks in science (England) 
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Source: TIMSS 2019. 
Note 1: The 1995 score is an average across the performance of year 8 and year 9 
pupils as the 1995 cycle assessed pupils across both year groups. 
Note 2: Response rates for TIMSS in England were relatively low in 1995, 1999 and 
2003. 


81 See Section 2.2 and Appendix C for descriptions of the international benchmarks. See Appendix D for a 
guide to interpreting the benchmark charts. 
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4.3.2 How did year 9 pupils in England perform in science relative to 
their peers in all other TIMSS countries? 


Thirty-nine countries participated in TIMSS 2019 year 9 science assessments. Full 
international analyses of their performance can be found in the TIMSS International 
Report 2019. 


In 2019, pupils in 9 countries performed significantly above England; 7 performed at a 
similar level and 22 significantly below. All of the East Asian countries, except Hong 
Kong, continued to perform significantly above England in 2019. Four more countries 
performed significantly above England in 2019 than in 2015. Both Australia and Lithuania 
joined this group owing to significant improvements in their performance, having 
performed significantly below England in 2015. The performance of pupils in Hungary 
and Russia was similar to 2015 but they also joined this group owing to England’s 
relative decline in 2019. 


There was some consistency in the list of countries performing similarly to England 
across the 2 cycles with Hong Kong, Ireland and the United States remaining in this 
group. Israel, Sweden and Turkey joined this group owing to England’s relative decline in 
2019, having performed below England in 2015. Portugal had not participated in the 
previous cycle. 


Tables 16, 17 and 18 below show how England’s year 9 pupils performed, in 2015 and 
2019, relative to those in a selection of other countries by average scale score. England’s 
average scale score was 537 in 2015 and 517 in 2019. 


Table 17: Year 9 science: all countries performing significantly above England in 


2015 and 2019 (average scores) 


2015 


2019 


Singapore (597) 

Japan (571) 

Chinese Taipei (569) 
Republic of Korea (556) 
Slovenia (551)** 


Singapore (608) 
Chinese Taipei (574) 
Japan (570) 

Republic of Korea (561) 
Russia (543) 

Finland (543)* 
Lithuania (534) 
Hungary (530) 
Australia (528) 


15 


Table 18: Year 9 science: all countries performing at a similar level to England in 


2015 and 2019 (average scores) 


2015 


2019 


Hong Kong (546) 
Russia (544) 
England (537) 
Kazakhstan (533) 
lreland (530) 
United States (530) 
Hungary (527) 


Ireland (523) 
United States (522) 
Sweden (521) 
Portugal (519) 
England (517) 
Turkey (515) 

Israel (513) 

Hong Kong (504) 


Table 19: Year 9 science: comparator countries performing significantly below 
England in 2015 and 2019, including countries that performed similarly to England 


in 2015 (average scores) 


New Zealand (513) 
Australia (512) 
Norway (509) 

Italy (499) 

and 21 others 


2015 2019 
Canada (526)** Italy (500) 
Sweden (522) New Zealand (499) 
Lithuania (519) Norway (495) 


France (489)* 
and 18 others 


Source: TIMSS 2019. 


Note 1. Asterisk (*) denotes countries that did not participate in the 2015 TIMSS year 9 


science assessments. 


Note 2. Double asterisk (**) denotes countries that did not participate in 2019 TIMSS year 


9 science assessments. 


4.3.3 How did year 9 pupils in England perform in science relative to 
their peers in the comparator countries? 


TIMSS scores 


In this section, trends are shown for countries with at least 3 cycles of assessment data 
since 2003. In some cases, countries have been selected so that no more than 5 
comparator countries are shown in each figure for clarity of presentation. Where this is 
the case, selection has primarily focused on the length of trend data available to draw 
comparisons with the performance of pupils in England over time. 
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In 2019, Hong Kong saw a significant decrease in its pupils’ average achievement in 
science. However, it has been retained as a highest-performing country due to prior 
performance. 


The performance of year 9 pupils in England in 2019 was significantly below the 
performance of year 9 pupils in 2003. Apart from pupils in Chinese Taipei and Hong 
Kong, the performance of those in all other countries shown in Figure 33 below was 
significantly higher in 2019 than 2003. Hong Kong’s trend has fluctuated more than those 
of other countries, with significant improvement in 2015 but significant decreases in 
average achievement in 2007 and 2019. Russia’s and Singapore’s trends both reflect the 
significant improvements gained in 2 cycles and no significant decreases. Pupils in all of 
these countries, with the exception of Hong Kong, performed significantly above their 
peers in England in 2019. 


Figure 33: Trends in year 9 science performance between 2003 and 2019 for 
England and highest-performing countries 
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Source: TIMSS 2019. 


Note 1: Comparator countries are only included if they have at least 3 cycles of data since 2003. 


Like England, the performance of year 9 pupils in New Zealand in 2019 was significantly 
below their performance in 2003. Performance in Australia and the United States in 2019 
was not significantly different from that in 2003. However, Australia’s 2019 performance 
was significantly higher than in 2015 and significantly above that of England. Figure 34 
below shows these trends. 
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Figure 34: Trends in year 9 science performance between 2003 and 2015 for 
England and English-speaking comparator countries 
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Note 1: Comparator countries are only included if they have at least 3 cycles of data 
since 2003. 

Note 2: New Zealand did not participate in year 9 TIMSS in 2007, represented by the 
dashed line. 


Of the 3 European comparators included in Figure 35 below, pupils in Italy and Lithuania 
performed significantly higher in 2019 than in 2003. Italy’s trend has been stable since 
2007, while the same is broadly evident for Sweden apart from a significant improvement 
in 2015. Similarly, Lithuania’s trend has been of stability across 4 cycles followed by a 
significant improvement in 2019. Including countries from the wider European comparator 
group, in 2019, pupils in England performed significantly below their peers in Finland and 
Lithuania. England’s pupils performed similarly to their peers in Sweden and significantly 
above their peers in the remaining countries. 
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Figure 35: Trends in year 9 science performance between 2003 and 2019 for 
England and European comparator countries 
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Note 1: Comparator countries are only included if they have at least 3 cycles of data 
since 2003. 


Figure 36 below shows the range of year 9 science scores in England against the 
countries from the 3 comparator groups from the 5th percentile (lowest-performing pupils) 
to the 95th percentile (highest-performing pupils) on the distribution of scores in each 
country. 


Year 9 pupils at the lower end of the distribution (the 5" percentile) achieved an average 
score of 399 in 2015 and 356 in 2019, a decrease of 43 scale points. At the top end of 
the distribution (the 95" percentile) the decrease was smaller, with pupils achieving an 
average score of 665 in 2015 and 658 in 2019, a decrease of 7 scale points. This larger 
decrease in performance for lower-achieving pupils, combined with a smaller decrease 
for the higher-achieving pupils, meant the achievement gap was greater in 2019 (302) 
than 2015 (266) by 36 scale points. Referring back to Figure 32, it can be seen that the 
increase in the range of scores in England between 2015 and 2019 was driven by this 
change at the bottom end of the distribution, with more pupils failing to achieve the low 
benchmark for year 9 science. 


By contrast, in the highest-performing group, most countries’ ranges were relatively small 
compared to England’s. Of the English-speaking countries, New Zealand’s and the 
United States’ were larger than England’s, while Australia and Ireland’s were smaller. 
England’s range was larger than those of all of the European comparator countries, with 
Italy’s the smallest. 
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Figure 36: Range of year 9 science achievement between the lowest and highest- 
achieving pupils across comparator countries 
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Source: TIMSS 2019. 
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TIMSS international benchmarks 


Fewer pupils in England reached each benchmark compared to their peers in the 
highest-performing group of countries, except in Hong Kong. For example, more than 4 
times as many year 9 pupils in the highest performing country, Singapore, reached the 
advanced benchmark compared to those in England (48% compared with 11%). 
However, in Russia, the difference was much smaller, with 13% of pupils reaching the 
advanced benchmark compared with 11% in England. In Singapore, just over twice as 
many pupils reached the high benchmark or above (77% compared with 38%). In 
addition, 91% of pupils in Singapore reached the intermediate benchmark or above, 
compared with 69% of pupils in England. Fewer pupils in England reached the low 
benchmark or above compared with their peers in all these countries, except Hong Kong. 


Nevertheless, compared with the international median across all participating countries, a 
larger share of pupils in England reached each benchmark”. 


Fewer pupils in England reached each benchmark than in 3 of the English-speaking 
countries, Australia, Ireland and the United States, with 2 exceptions: more pupils in 
England reached the low benchmark or above than their peers in the United States and 
more reached the advanced benchmark than their peers in Ireland. More of England’s 
pupils reached all benchmarks than was the case for pupils in New Zealand. 


Fewer pupils in England reached the advanced benchmark than those in Finland and 
Sweden from the European comparator countries, while fewer in England reached the 
high benchmark or above than pupils in Finland, Lithuania and Sweden. More pupils in 
England reached these benchmarks than in the remainder of the European comparator 
countries. 


Similarly, fewer pupils in England reached the intermediate benchmark or above 
compared to their peers in Finland, Lithuania and Sweden, with the reverse the case in 
comparison to pupils from France, Italy and Norway. Fewer pupils in England reached 
the low benchmark or above than was the case for their peers in Finland, Italy and 
Lithuania, with the same percentage as in England reaching this benchmark or above in 
Sweden. More pupils in England reached these benchmarks than was the case for pupils 
in the remaining countries, France and Norway. 


Figure 37 below presents these data. Data on all other participating countries are 
available in the T/IMSS International Report 2019. 


62 International medians rather than international averages are calculated for this data set. 
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Figure 37: Percentage of year 9 pupils reaching the international benchmarks in 
science (England and comparator countries) 
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Source: TIMSS 2019. 
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4.4 What does TIMSS tell us about pupil progress in 
science between years 5 and 9? 


As the target year 9 cohort in 2019 was the same as the cohort of pupils who were in 
year 5 in 2015, TIMSS allows for comparison of relative progress achieved by the cohort 
between grades. However, due to the sampling approach (see section 1.2), the year 5 
pupils who took the assessments in 2015 were from the same cohort but were not 
necessarily the same pupils as those sampled in year 9 in 2019 The assessments taken 
by year 5 and year 9 pupils, and frameworks from which these were taken, were also 
different. 


As shown in Figure 38 below, the average score of the year 5 cohort in 2015 in England 
was significantly above the TIMSS centrepoint (536 compared with 500). By the time this 
cohort reached year 9 in 2019, their average performance was 17 points above the 
TIMSS centrepoint (517 compared with 500). While still significantly above the TIMSS 
centrepoint, this reflects a decrease of 19 scale points in terms of relative performance 
against this centrepoint (536 compared with 517). This is in contrast to 2015 when 
relative performance showed an increase of 8 scale points (a year 5 average score of 
529 compared with a year 9 average score of 537). 


Across the comparator countries in 2019, decreases similar to the one seen for England 
were found in most cases. Of the highest-performing countries for year 5 science in 
2015, half (Hong Kong, the Republic of Korea and Russia) did not secure a higher level 
of achievement in year 9 in 2019. In the case of Hong Kong, year 9 performance declined 
significantly by 42 scale points. Of the English-speaking countries, only Australia reached 
a higher level of achievement, and of the European comparator countries only Lithuania 
and France did so. 
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Figure 38: A comparison of the science performance of year 5 pupils in 2015 and 
year 9 pupils in 2019 (England and other countries from the comparator groups) 
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Chapter 5. Mathematics and science performance in 
subject and cognitive domains 


TIMSS enables a detailed comparison of pupils’ mathematics and science performance 
in specific subject and cognitive domains. Each of the assessment questions is 
categorised according to the area of the curriculum it covers (referred to in TIMSS as 
content domains) and the different cognitive skills it requires (referred to in TIMSS as 
cognitive domains)®°. 


In year 5 mathematics, there were 3 content domains: Data; measurement and 
geometry; and number (in 2015, these were entitled data display; geometric shapes and 
measures; and number). In year 9, there were 4: Algebra; data and probability 
(previously entitled data and chance); geometry; and number. 

In year 5 science, there were 3 domains: Earth science; life science; and physical 
science. In year 9, there were 4: biology; chemistry; Earth science; and physics 

The domain names for science have remained consistent from 2015. 


To assess the relative strengths and weaknesses of year 5 and 9 pupils across the 
different TIMSS mathematics content domains, our analysis compared their average 
score in each domain: 


to the TIMSS international average 

across TIMSS cycles 

to England’s overall average score 

to the performance of other comparator group countries 


The comparator countries referred to in this chapter are listed in section 1.5%. 


5.1 Main findings 


Where the terms ‘strongest’ and ‘weakest’ are used in the following statements and 
throughout this chapter, they denote that the average score for the domain in question is 
either significantly above or below the relevant country’s overall mathematics or science 
average score. When comparing average score differences between countries, reference 
is made to average scores being comparatively higher or lower; this does not mean such 
differences are statistically significant. 


Mathematics 
e In 2019, pupils in England performed above the international averages in all 
content domains (apart from algebra for year 9) and all cognitive domains (apart 
from knowing for year 9). 
e In 2019, the performance of year 5 pupils in England was strongest in the data 
and number domains, and weakest in the measurement and geometry domain. 


63 See the TIMSS 2019 Frameworks: Mullis, |. V. S., & Martin, M. O. (Eds.). (2017). TIMSS 2019 
Assessment Frameworks. Retrieved from Boston College, TIMSS & PIRLS International Study Center 


website: http://timssandpirls.bc.edu/timss2019/frameworks/ 
64 Canada, Ireland, Germany, the Netherlands Poland and Spain did not participate in the year 9 study. 
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There was a significant improvement in the performance of England’s year 5 
pupils in the number and data domains in 2019. In the measurement and 
geometry domain, performance was similar to that recorded in 2015, with a non- 
significant increase in 2019. 

Year 9 pupils in England were strongest in data and probability and in number, 
and weakest in algebra and geometry. These relative strengths and weaknesses 
in 2019 mirrored the 2015 outcomes. 

In 2019, year 5 pupils’ strength in data in England was in contrast to the majority 
of the highest-performing countries, which performed strongly in measurement 
and geometry; again this mirrored the 2015 findings. 

Similarly, in year 9, while pupils in England performed strongly in data and 
probability, the strengths of pupils in the highest-performing countries tended to lie 
across the algebra and geometry domains. This was a change from the previous 
cycle when there was no clear pattern in relative content domain strengths in 
these countries. 

Year 5 pupils in England were strongest in the knowing cognitive domain in 
mathematics in 2019, as they were in 2015. 

In 2019, as in 2015, England’s year 9 pupils were weakest in the knowing 
cognitive domain. They were strongest in the applying domain, whereas in 2015, 
they were strongest in reasoning. 


Science 


In 2019, pupils in England performed above the international averages in all 
science content and cognitive domains in both years 5 and 9. 

In 2019, the performance of year 5 pupils in England in the physical and life 
science domains was the same as the overall science average score for pupils in 
England. Their performance was weakest in Earth science. 

Year 5 performance in 2019 was similar to that recorded in 2015 across all the 
content domains, although the physical science score was not significantly above 
the overall score for science in 2019, as it was in 2015. 

In 2019, the performance of year 9 pupils across all content domains (biology, 
chemistry, physics and Earth science) was in line with England’s overall science 
average score. 

Year 9 pupils’ average scores in all content domains were significantly lower in 
2019 than in 2015. This was in line with England’s lower overall science average 
score in 2019. 

In year 5, pupils in 5 out of the 6 highest-performing countries achieved their 
highest average scores in the physical science domain, with these scores being 
significantly above their overall science average scores. In year 9 there was no 
clear pattern of relatively strong domain scores for pupils in the highest-performing 
countries. 

Year 5 pupils’ performance in England was strongest in the knowing and 
reasoning cognitive domains and weakest in the applying domain in 2019. In 
2015, there were no significant differences between average scores for each 
cognitive domain and the overall science average score. 

In 2019, year 9 pupils in England performed in line with their science overall 
average score in the knowing and applying cognitive domains but significantly 
below this in the reasoning domain. This was in contrast to 2015 when reasoning 
was the strongest cognitive domain. 
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5.2 How did pupils in England perform across different content 
domains? 


5.2.1 How did year 5 pupils in England perform across 
different mathematics content domains? 


In 2019, year 5 pupils in England performed significantly above the international 
averages in all 3 domains. 


In 2019, as in 2011 and 2015, year 5 pupils in England performed significantly above the 
overall mathematics average score in data (see Figure 39 below). In 2015 and 2019, they 
performed significantly below their overall mathematics average score in measurement 
and geometry. While in 2011 number had been the weakest domain (539 compared to an 
average score of 542), in 2019 pupils performed significantly above their overall 
mathematics average score in this domain (559 compared to an average score of 556). 


In 2019, year 5 pupils in England performed significantly above their corresponding 
scores in both 2011 and 2015 in the number and data domains. In the measurement and 
geometry domain, relative differences between the 2019 and previous cycle scores in 
2011 and 2015 were not significant. 


Figure 39: Average scores for 2011-2019 in different mathematics content domains 
compared to the overall mathematics average scale score (England, year 5) 
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Source: TIMSS 2019. 


Note 1: Asterisks (*) indicate domain average scores that were significantly different from 
England’s overall average score. 
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Even in their strongest domains, year 5 pupils in England tended to achieve lower 
average scores in 2019 compared to their peers in the highest-performing countries, in 
line with the relative performance in overall mathematics average scores. The exception 
was the data domain, where the average score for pupils in England was above that of 
pupils in Russia (565 compared to 560). This was a change from 2015, when England’s 
performance in this domain was 21 scale points below Russia’s. The change was due to 
a significant (13 scale-point) increase in England’s year 5 pupils’ performance in this 
domain in 2019 compared with 2015, combined with a corresponding significant (13- 
point) decrease in Russia’s pupils’ performance. 


In 5 of the 6 highest-performing countries, pupils’ average scores in the measurement 
and geometry domain were significantly above their overall mathematics average scores. 
The exception was Singapore, where pupils had a relative strength in the number 
domain. Only in Japan was pupils’ average score in the data domain significantly above 
their overall mathematics average score, as in England. However, unlike in England, 
Japan’s pupils also performed above their overall mathematics average score in 
measurement and geometry. 


Pupils’ average scores in England were higher in all domains than those of their peers in 
all other English-speaking countries, except Northern Ireland. Northern Ireland was the 
only country in this group whose pupils’ overall mathematics average score was 
significantly above England’s in 2019. Ireland was the only country where pupils 
performed at a similar level to England’s in 2019 and, like England, its pupils’ average 
score for the number domain was significantly above the overall mathematics average 
score, while their score for the measurement and geometry domain was significantly 
below. England’s pupils’ average score in the data domain was also significantly above 
their overall mathematics average score, whereas those in Ireland’s average score for 
data was significantly below their overall average. Northern Ireland’s pupils’ average 
scores in the number, and measurement and geometry domains were higher than those 
for pupils in England, while its pupils performed similarly in the data domain. 


Pupils’ average scores in England were higher in all domains than those of their peers 
from the European comparator countries, except in 1 case. The average score for pupils 
in Norway in the measurement and geometry domain was similar to that of their peers in 
England, despite the overall mathematics average score for pupils in Norway (543) being 
significantly below that of pupils in England (556). 


5.2.2 How did year 9 pupils in England perform across 
different mathematics content domains? 


As in year 5, year 9 pupils in England achieved average scores significantly above the 
TIMSS international averages in the number, geometry and data and probability domains 
in 2019. In 2019 England’s year 9 pupils’ algebra score was in line with the international 
average for this domain. There was a 12 scale-point increase in the year 9 algebra score 
for England between 2015 and 2019 but this increase was not significant. 


As in 2011 and 2015 (shown in Figure 40 below), year 9 pupils in England performed 


most strongly in both the number (519) and data and probability (523) domains in 2019. 
Both average scores were significantly above their overall mathematics average score for 
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year 9 mathematics in England (515). By contrast, the average scores for the algebra 
(504) and geometry (509) domains were significantly below their overall average score. 


The average score in 2019 for the domain of relative strength, data and probability, was 
significantly below scores achieved in 2011 and 2015. No other differences between 
domain scores in 2019 and the previous cycles of 2011 and 2015 were significant. The 
only domain in which pupils’ average score in 2019 was not lower than in 2015 was 
algebra. However, as stated above, this increase between 2015 and 2019 was not 
significant. 


Figure 40: Average scores for 2011-2019 in different mathematics content domains 
compared to the overall mathematics average scale score (England, year 9) 
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Source: TIMSS 2019. 


Note 1: Asterisks (*) indicate domain average scores that were significantly different from 
England’s overall average score. 


In 2019, with 1 domain exception, year 9 pupils in England achieved lower average 
scores compared to their peers from the highest-performing comparator countries, even 
in their strongest domains. The exception, as in 2015, was the data and probability 
domain, where the average score for England’s pupils was above that of their peers in 
Russia (523 compared to 517). Singapore was the only country in this group where 
pupils’ average score was highest in the data and probability domain, as in England. 
Singapore was also the only country from this group where this domain score was 
significantly above the overall mathematics average score. Pupils’ average scores in the 
remaining East Asian countries were highest in geometry, while pupils’ average scores in 
Russia were highest in algebra. 


Pupils’ average scores in all of the highest-performing countries were significantly higher 
in the algebra domain than their overall mathematics average scores in 2019. This 
outcome was also the case in the geometry domain, with the exception of pupils in 
Russia. 


Compared to the 4 other English-speaking countries, year 9 pupils in England achieved a 
higher average score in the geometry domain compared to their peers, except in 1 case: 
pupils’ average score in Australia for geometry (513 compared to 509). 


As in England, pupils’ average scores in all of these countries, apart from the United 
States, were highest in data and probability, where they were significantly above their 
overall mathematics average scores. In the United States, pupils’ average scores were 
highest in both the number and algebra domains, with each significantly above their 
overall mathematics average score. Apart from in New Zealand, pupils’ average scores in 
all these countries were significantly above their overall mathematics average scores in 
the number domain, as in England. Pupils’ average scores for geometry in all of these 
countries were significantly below their overall mathematics average scores. 


In the number and data and probability domains, the average scores for year 9 pupils in 
England were higher than those of their peers from any of the European comparator 
countries. This was also the case for algebra, with the exception of pupils in Lithuania. 
Similarly to the case in England, pupils’ average scores in the algebra domain were 
significantly below the overall mathematics average score in all 6 countries. In France, 
Italy and Lithuania, pupils’ average scores in the geometry domain were significantly 
above their overall mathematics average score. 


5.2.3 How did year 5 pupils in England perform across 
different science content domains? 


As in 2011 and 2015, year 5 pupils in England performed significantly above the 
international averages in all 3 content domains. 


As shown in Figure 41 below, year 5 pupils in England achieved the same average score 
for the life science and physical science domains in 2019 (both of which were the same 
as England’s overall science average score of 537). In 2011 and 2015, average scores in 
the life science domain were not significantly different from the overall average science 
scores, while scores in the physical science domain were significantly above the overall 
average scores in both years. As in 2011 and 2015, pupils in 2019 were weakest in Earth 
science, the average score for which was significantly below their overall science 
average score (533 compared to 537). 


Pupils’ performance in 2019 compared to the previous 2 cycles (2011 and 2015) was not 


significantly different across content domains, apart from in Earth science, where 
achievement was significantly above the 2011 score (533 compared to 522). 
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Figure 41: Average scores for 2011-2019 in different science content domains 
compared to the overall science average scale score (England, year 5) 


537 
Overall 536 
529 


Life Science 536 


Physical * 
Science a4 
535* 
Earth e 
Science 527 
522* 


470 490 510 530 550 570 


2019 =2015 =2011 


Source: TIMSS 2019. 


Note 1: Asterisks (*) indicate domain average scores that were significantly different from 
England’s overall average score. 


Year 5 pupils’ average scores in England were lower across all domains in 2019 than 
those of their peers in the highest-performing group of countries, apart from Hong Kong, 
where pupils’ overall science average score was similar to those of pupils in England. 
With the exception of Hong Kong, pupils’ average scores in these countries were highest 
in physical science, with average scores in this domain significantly above their overall 
science average scores. 


The average score for England’s year 5 pupils in the physical science domain was higher 
than those of their peers in any of the other English-speaking countries. Only in Australia 
and the United States were pupils’ average scores in the life science domain higher than 
for their peers in England, and only in Ireland and the United States was this the case in 
the Earth science domain. The average scores for pupils in Ireland and Northern Ireland 
were highest in the Earth science domain, while in the remaining countries, pupils’ 
average scores were highest in the life science domain. 


Year 5 pupils in England achieved higher average scores in at least 1 science domain 
compared to their peers in 6 out of 10 European comparator countries. Finland’s pupils’ 
average scores were higher than those of pupils in England in all 3 domains; Sweden’s 
and Norway’s pupils scored higher averages in 2 domains (life science and Earth 
science) and Lithuania’s pupils’ averages were higher in physical science. Pupils’ 
average scores in most of these European comparator countries were lowest in the 
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physical science domain. However, this was not the case in England, Germany and 
Lithuania, where the domain with the lowest average score was Earth science. 


5.2.4 How did year 9 pupils in England perform across 

different science content domains? 

As in 2011 and 2015, year 9 pupils in England performed significantly above the 
international average in all of the content domains. In all domains, year 9 pupils’ average 
scores in 2019 were significantly below those achieved in both 2011 and 2015. This was 
in line with pupils’ overall science average score being lower. 

As shown in Figure 42 below, in 2019, England’s year 9 pupils achieved the same or 
similar average scores in the Earth science, biology and physics domains as their overall 
science average score. They scored lowest in chemistry (512), although this was not 
significantly different from the overall science average score. 


Figure 42: Average scores for 2011-2019 in different science content domains 
compared to the overall science average scale score (England, year 9) 
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Source: TIMSS 2019. 


Note 1: Asterisks (*) indicate domain average scores that were significantly different from 
England’s overall average score. 


Year 9 pupils in England achieved lower average scores compared to their peers in the 
highest-performing group of countries, apart from Hong Kong, across all the domains. 
However, this was in line with England’s and Hong Kong’s lower overall science average 
scores. Hong Kong performed at a similar level to England in terms of the overall science 
average score and its pupils’ average scores for all domains were below those of pupils 
in England. 
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There was no clear pattern of relatively strong domain scores across this group of 
countries. While pupils’ average scores in the chemistry domain were significantly above 
the overall science average score in Chinese Taipei, Russia and Singapore in 2019, the 
reverse was the case for pupils in Japan and the Republic of Korea. 


Year 9 pupils in England achieved lower average scores in each domain compared to 
their peers in Australia, which was the English-speaking country with the highest overall 
science average score. Their average scores were also lower compared with Ireland’s 
pupils in all domains except chemistry, where they were the same. Pupils in England 
achieved similar average scores to their peers in the United States in the biology, 
chemistry and physics domains but had a lower average score in Earth science. 
England’s pupils achieved higher average scores than pupils in New Zealand in all 
domains. Although the average score of England’s pupils was lowest in the chemistry 
domain, this was not significantly different from their overall science average score. 
However, pupils in Australia, Ireland and the United States all had significantly lower 
scores in this domain compared to their overall science average scores. All 4 other 
English-speaking comparator countries saw pupils achieve significantly higher Earth 
science scores compared to their overall science average scores, while in England 
scores in this domain were in line with this overall average. 


In 2019, pupils in England achieved lower average scores in each domain compared to 
their peers in Finland and Lithuania. Pupils in England also achieved lower average 
scores compared to their peers in Sweden in all domains except chemistry, in which 
pupils in Sweden had scores significantly below their overall science average score. In 
relation to the remaining European comparator countries, England’s pupils achieved 
higher average scores in each domain than pupils in Italy, Norway and France, with the 
exception of Earth science in Norway. In all countries except Lithuania, pupils’ average 
scores in the Earth science domain were significantly above their overall science average 
scores, whereas in England pupils’ performance in this domain was the same as this 
overall average. 


5.3 How did pupils in England perform in 
different cognitive domains? 


In both mathematics and science, TIMSS assesses pupils’ performance in 3 cognitive 
domains: knowing, applying and reasoning. The domains describe the kind of thinking 
that pupils do when engaged with both mathematics and science tests, although with 
different emphases depending on the subject and year group®. For example, there is 
more emphasis on the knowing domain in year 5 science, but a greater emphasis on the 
applying domain in year 9 science, and in year 5 and 9 mathematics. Pupil performance 
in the 3 cognitive domains is highly correlated with performance in the TIMSS subject 
domains and performance overall, meaning that no one domain is more or less important 
for overall performance. 


85 See the TIMSS 2019 Frameworks: Mullis, |. V. S., & Martin, M. O. (Eds.). (2017). TIMSS 2019 
Assessment Frameworks. Retrieved from Boston College, TIMSS & PIRLS International Study Center 
website: http://timssandpirls.bc.edu/timss2019/frameworks/ 
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The descriptions of the 3 domains differ slightly between mathematics and science; 
broadly they are described as encompassing the following: 


e knowing: the facts, concepts, and procedures pupils need to know 

e applying: pupils’ application of knowledge and understanding to, for example, 
solve problems and answer questions 

e reasoning: goes ‘beyond the solution of routine problems’ in mathematics and, in 
science ‘includes using evidence and science understanding to analyse, 
synthesize, and generalize’ — in both subjects there is an emphasis upon doing 
these within ‘unfamiliar situations and complex contexts’® 


5.3.1 How did year 5 pupils in England perform in the mathematics 
cognitive domains? 


In 2019, year 5 pupils in England performed significantly above the international average 
in all 3 cognitive domains for mathematics. 


As shown in Figure 43 below, year 5 pupils in England were strongest in the knowing 
domain in 2019, with an average score of 563; this was significantly above England’s 
overall average score for mathematics (556). The average score for reasoning was 
significantly above that in 2011 and 2015, but not significantly different from the overall 
mathematics average score in 2019. Average scores for knowing and applying in 2019 
were also above those achieved in 2015, but the difference was not significant. However, 
average scores were significantly above those achieved in 2011 in these 2 domains. 


66 See the TIMSS 2019 Frameworks: Mullis, |. V. S., & Martin, M. O. (Eds.). (2017). TIMSS 2019 
Assessment Frameworks. Retrieved from Boston College, TIMSS & PIRLS International Study Center 
website: http://timssandpirls.bc.edu/timss2019/frameworks/ 
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Figure 43: Average scores for 2011-2019 in mathematics cognitive domains 
compared to the overall mathematics average scale score (England, year 5 
mathematics) 
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Source: TIMSS 2019. 


Note 1: Asterisks (*) indicate domain average scores that were significantly different from 
England’s overall average score. 


With the exception of those in Hong Kong and Russia, pupils in the highest-performing 
group of countries were strongest in the knowing domain, with scores significantly above 
their overall mathematics scores. In 4 out of 6 of these countries, pupils’ average scores 
were lowest in the reasoning domain: the exceptions were the Republic of Korea and 
Russia. 


Of the English-speaking group, in all of the countries with the lowest overall mathematics 
average scores — Australia, Canada and New Zealand — pupils’ average scores were 
lowest in the knowing domain. In Northern Ireland (the country with the highest overall 
average score for mathematics), pupils’ average score was highest in the knowing 
domain. In Ireland and the United States, pupils’ average scores in the knowing and 
applying domains were above those for the reasoning domain. 


Of the 10 European comparator countries, the 6 with the highest overall mathematics 
average scores had their highest average scores in the reasoning domain, with the 
exception of Lithuania. Only in France — the lowest-performing country from this group in 
terms of its overall mathematics average score — was pupils’ average score highest in the 
knowing domain. 
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5.3.2 How did year 9 pupils in England perform in the mathematics 
cognitive domains? 


In 2019, year 9 pupils in England performed significantly above the international 
averages in the applying and reasoning domains. Pupils in England performed in line 
with the international average in the knowing domain for year 9 mathematics. 


As shown in Figure 44 below, year 9 pupils in England were strongest in the applying 
domain in 2019 with an average score of 518, significantly above their overall 
mathematics average score (515). In contrast to year 5 pupils, year 9 pupils’ average 
score for knowing was significantly below their overall mathematics average score, as it 
was in 2011 and 2015. None of the 2019 average scores were significantly different from 
those achieved in 2011 or 2015. 


Figure 44: Average scores for 2011-2019 in mathematics cognitive domains 
compared to the overall mathematics average scale score (England, year 9 
mathematics) 
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Source: TIMSS 2019. 


Note 1: Asterisks (*) indicate domain average scores that were significantly different from 
England’s overall average score. 


In none of the highest-performing countries were year 9 pupils’ average mathematics 
scores highest in the applying domain. In Hong Kong, the Republic of Korea and 
Singapore, pupils’ average scores in this domain were significantly below their overall 
mathematics average scores in 2019. In the remaining 3 countries, pupils’ average 
scores for the applying domain were in line with their overall averages. This was in 
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contrast to England in 2019, where pupils were strongest in this domain. There are no 
clear patterns of domains in which these countries perform best. 


Similarly to England, in both Australia and New Zealand from the English-speaking 
countries, pupils’ average scores were strongest in the applying domain and weakest in 
the knowing domain, with performance significantly different from their overall 
mathematics average scores in 2019. By contrast, in Ireland and the United States, 
pupils’ average scores were strongest in the knowing domain, both significantly above 
their overall mathematics average scores. 


As in England, in none of the European comparator countries were year 9 pupils’ 
mathematics average scores highest in the knowing domain, and with the exception of 
Lithuania, pupils’ average scores in this domain were significantly below their overall 
mathematics average scores in 2019. 


5.3.3 How did year 5 pupils in England perform in the science cognitive 
domains? 


In 2019, year 5 pupils in England performed significantly above the international average 
in all of 3 cognitive domains for science. 


As shown in Figure 45 below, year 5 pupils in England were equally strong in the 
knowing and reasoning domains in 2019, with average scores of 544. These were 
significantly above their overall science average score (537). However, performance in 
the applying domain was significantly below their overall science average score. The 
average score for knowing was significantly above those achieved by England’s pupils in 
2011 and 2015, while the average score for applying in 2019 was significantly below that 
achieved in 2015. Pupils’ average score in the reasoning domain in 2019 was 
significantly above that achieved in 2011. 
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Figure 45: Average scores for 2011-2019 in science cognitive domains compared 
to the overall science average scale score (England, year 5 science) 
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Source: TIMSS 2019. 


Note 1: Asterisks (*) indicate domain average scores that were significantly different from 
England’s overall average score. 


Among the highest-performing countries in 2019, only in Hong Kong was year 5 pupils’ 
average score highest in the knowing domain in science. With the exception of Hong 
Kong and Singapore, pupils’ performance in the applying domain in these countries was 
significantly above their overall science average scores in 2019. This was in contrast to 
England, for which pupils’ performance in this domain was significantly below the 
country’s overall score for year 5 science. In both Japan and Singapore, as in England, 
pupils’ average scores in reasoning were significantly above their overall science 
average scores. 


As in England pupils’ average scores in the applying domain in all of the other English- 
speaking countries, apart from in Ireland, were significantly below their overall science 
average scores in 2019. In all these countries, as in England, pupils’ average scores 
were highest in either the knowing or reasoning domains. Apart from in Canada and 
Northern Ireland, pupils’ average scores in the knowing domain, as in England, were 
significantly above their overall science average scores. 


Of the European comparator group, only in France and Poland were pupils’ average 
scores highest in the applying domain. In the countries with the same or higher overall 
science average score to England’s, pupils’ average scores for the applying domain were 
significantly below their overall science average scores and this was the same for 
England’s pupils. 
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5.3.4 How did year 9 pupils in England perform in the science cognitive 
domains? 


In 2019, year 9 pupils in England performed significantly above the international average 
in all of 3 cognitive domains for science. 


As shown in Figure 46 below, in 2019, year 9 pupils’ average scores for the knowing 
(520) and applying (515) domains were not significantly different from the overall science 
average score (517). However, the average score for reasoning was significantly below 
their overall science average score (513 compared to 517). 


In 2019, the average score differences in both the applying and reasoning domains were 
significantly below those achieved in both 2011 and 2015. For applying, the difference 
between 2019 and 2015 was 24 scale points®’ and for reasoning the difference was 31 
scale points. These reflect the lower performance in 2019 in the overall science average 
score. Whereas pupils were strongest in the reasoning domain in 2015, this was their 
weakest domain in 2019. Year 9 pupils’ average score for the knowing domain in 
England was below the average scores achieved in this domain in 2011 and 2015, but 
the differences were not significant. 


Figure 46: Average scores for 2011-2019 in science cognitive domains compared 
to the overall science average scale score (England, year 9 science) 
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Source: TIMSS 2019. 
Note 1: Asterisks (*) indicate domain average scores that were significantly different from 
England’s overall average score. 


6” This figure differs from the chart due to rounding. 
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In the highest-performing countries there were no clear patterns of relative domain 
strengths or weaknesses. In both Chinese Taipei and Singapore, pupils’ average scores 
in the knowing domain were significantly above their overall science average score but 
their scores for the reasoning domain were significantly below this. In Japan, pupils’ 
average score for the applying domain was significantly above the overall science 
average score, while for pupils in the Republic of Korea, this was the case for reasoning. 
In Russia, pupils’ average scores for each domain were in line with their overall science 
average score. 


In all of the other English-speaking countries, pupils in 2019 were strongest in the 
reasoning domain, with average scores significantly above their overall science average 
scores. This was in contrast to England where pupils were weakest in this domain. In the 
knowing domain, pupils in England performed in line with their overall science average 
score. This domain was an area of weakness for the other English-speaking comparators 
with pupils’ average scores significantly below their overall science scores. 


In the European comparator group, year 9 pupils in Finland, France and Lithuania were 
strongest in the reasoning domain, with average scores above their overall science 
average scores. There were no countries in this group with pupils’ average scores 
highest in the applying domain. Apart from in Italy, pupils in all these countries performed 
significantly below their overall science average scores in this domain. 
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Chapter 6. Mathematics and science performance by 
pupil characteristics 


Year 5 and 9 pupils’ performance in the TIMSS mathematics and science assessments 
can be analysed using 3 key pupil characteristics that are only available from the 
National Pupil Database (NPD): 


e ethnicity 
e first language 
e eligibility for free school meals 


The following sections present 2 ways of doing this: 


1. The average assessment scores by these key characteristics 
2. Achievement of key TIMSS benchmark scores by these key characteristics. 


TIMSS data also enable comparisons to be drawn between the performance of boys and 
girls in each subject in their respective year groups and to compare England’s gender 
differences with those of other countries. The comparator countries referred to in this 
chapter are listed in section 1.5%. 


While there are no international comparisons for ethnicity or first language, TIMSS 
achievement data have been matched to data from the NPD to provide comparisons 
among the achievement of ethnic groups in England. The 6 ethnic groups used in this 
chapter include those used by the Department for Education for its statistical releases. It 
should be noted that there may be some variation in achievement within different high- 
level ethnic groups. For example, the Asian group pupils from Indian, Bangladeshi, 
Pakistani and other Asian backgrounds, and other analysis shows that some of these 
groups perform better on average®’. The average scores for Chinese pupils as a 
separate category were not included in the 2019 report due to the sample size being too 
small for the analysis to be robust. 


The primary definition used in England for disadvantage is pupils’ past and current 
eligibility for free school meals (FSM). This measure is not something that is applied 
internationally; TIMSS asks pupils how many books they have at home in order to 
provide an indication of their socio-economic status, with fewer books being associated 
with lower socio-economic status. 


6.1 Main findings 


Gender 
e In 2019, year 5 boys’ average score was above year 5 girls’ average score in 
mathematics in England, although the difference was not significant. By contrast, 
in 2015, the difference between boys’ and girls’ average scores was significant. 


88 Canada, Ireland, Germany, the Netherlands Poland and Spain did not participate in the year 9 study. 
89 See, for example, DfE (2020) Key Stage 4 performance, 2019 (revised). Available at: 


https://www.gov.uk/government/statistics/key-stage-4-performance-2019-revised 
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e Boys outperformed girls in the majority of comparator countries in both year 5 and 
year 9, but the difference was only significant in about half of these. 

e Asin 2015, boys and girls performed similarly in year 9 mathematics in 2019. The 
picture was more mixed in comparator countries; in some, girls outperformed boys 
and in others, boys outperformed girls. On the whole, the differences were not 
significant. 

e In 2019, year 5 boys’ average score in science was the same as girls’ average 
score in England. This mirrored the 2015 study when their average scores were 
also the same. The majority of comparator countries saw boys outperforming girls, 
but the differences were largely non-significant. 

e Asin 2015, boys and girls performed similarly in year 9 science in 2019 in 
England; girls’ average score was just above boys’, but the difference was not 
significant. In the majority of comparator countries there was also no significant 
difference between boys’ and girls’ scores. 


Ethnicity 
e In mathematics for pupils in year 5 and year 9, there were no significant 
differences in performance between ethnic groups. 
e Inscience in both years 5 and 9 pupils from all other ethnic groups aside from 
Black pupils performed comparably to White British pupils; Black pupils scored 
significantly below White British pupils in 2019. 


Differences by first language 
e There were no significant differences in scores for pupils with and without English 
as a first language in mathematics for years 5 and 9, or in science for year 5. 
However, in year 9 science, pupils with English as a first language scored 
significantly higher than pupils whose first language was not English. 


Socio-economic status 

e Pupils who had been eligible for free school meals (FSM) at any time in the 
previous 6 years performed significantly lower across both year groups and both 
subjects than their peers who had not been eligible for FSM. 

e Using the TIMSS measure of number of books at home, a proxy for socio- 
economic status, there was a wide gap in performance at both year groups in both 
subjects between pupils who had very few books at home and those who had 
many. The achievement gap between the 2 groups was around 100 scale points. 


Achievement at the international benchmarks by pupil characteristics 

e In most cases, there were no significant differences for pupils reaching the 
international benchmarks by pupil characteristics. The main exception was for 
pupils who had been eligible for FSM in the last 6 years, of whom a significantly 
lower percentage reached each benchmark, for both year groups and both 
subjects, compared to their peers. 

e In year 5 science, significantly fewer Asian pupils reached the advanced 
benchmark than White British pupils; in year 9 science, significantly fewer pupils 
whose first language was not English reached the intermediate and high 
benchmarks or above than White British pupils; and in year 9 science significantly 
fewer Black pupils reached the low benchmark or above than White British pupils. 
There were no other significant differences between the performance of White 
British pupils and those from other ethnic groups against the benchmarks. 
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6.2 Does performance differ by gender, ethnicity and first 
language in year 5 mathematics? 


Figure 47 below presents the performance of different year 5 pupil groups alongside the 
overall mathematics scores for 2019 and 2015. 


In 2019 there was no significant difference between the average scores of year 5 boys 
(560) and girls (552) in mathematics in England. By contrast, in 2015, boys significantly 
outperformed girls on average. 


As in England in 2019, in half of both the highest-performing and English-speaking 
countries, there were no significant differences between boys’ and girls’ average 
achievement. However, boys’ average scores were significantly above girls’ average 
scores in several comparator countries, including the Republic of Korea, Russia, 
Singapore, Australia, Canada, the United States and 7 of the 10 European comparator 
countries. In none of the comparator group countries did girls perform significantly higher 
than boys. 


White British pupils achieved an average score of 548, which was not significantly 
different from the average scores for the other ethnic groups. 


The average score for year 5 pupils whose first language was not English (559) was not 


significantly different from the average score for pupils whose first language was English 
(549). 
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Figure 47: Average achievement in year 5 mathematics by gender, ethnicity and 
first language (England) 
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Source: NPD and TIMSS 2019. 


Note 1: The average scores for Chinese pupils as a separate category were not included 
in the 2019 report due to the sample size being too small for the analysis to be robust. 


6.3 Does performance differ by gender, ethnicity and first 
language in year 9 mathematics? 


Figure 48 below presents the performance of different year 9 pupil groups alongside the 
overall mathematics scores for 2019 and 2015. 


In 2019, the year 9 boys’ average score (516) was similar to the year 9 girls’ average 
score (514) in mathematics in England, as in 2015 when they were 520 and 517 
respectively. The differences were not significant in either cycle. 


In 2019, as in England, there were no significant differences between boys’ and girls’ 
performances in the highest-performing countries, in the other English-speaking 
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countries and in the majority of European comparator countries. However, boys’ 
performance was significantly above girls’ performance in France and Italy. As in year 5, 
in no comparator group country was girls’ performance significantly above boys’ 
performance. 


In 2019, there were no significant differences between the performance of White British 
pupils and pupils from the other ethnic groups. 


The average mathematics score for pupils whose first language was English (511) was 
not significantly different from the average score for pupils whose first language was not 
English (504). 


Figure 48: Average achievement in year 9 mathematics by gender, ethnicity and 
first language (England) 
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Source: NPD and TIMSS 2019. 


Note 1: The average scores for Chinese pupils as a separate category were not included 
in the 2019 report due to the sample size being too small for the analysis to be robust. 
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6.4 Does performance differ by gender, ethnicity and first 
language in year 5 science? 


Figure 49 below presents the performance of different pupil groups alongside the overall 
science scores for 2019 and 2015. 


In 2019, the year 5 boys’ average score in science was the same as the girls’ average 
score in England (537). This mirrored the 2015 study when their average scores were 
also the same (536). 


Figure 49: Average achievement in year 5 science by gender, ethnicity and first 
language (England) 
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Source: NPD and TIMSS 2019. 


Note 1: The average scores for Chinese pupils as a separate category were not included 
in the 2019 report due to the sample size being too small for the analysis to be robust. 
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In 2019, there were only 2 significant difference between boys’ and girls’ average scores 
in the highest-performing countries: boys performed significantly higher than girls in the 
Republic of Korea and Singapore. Boys also performed significantly higher than girls in 
Canada and the United States among the English-speaking countries, and in Italy among 
the European comparator countries. As in mathematics, in none of the comparator 
countries was girls’ performance significantly above boys’ performance. 


The average score for Black pupils (519) in 2019 was significantly below the average 
score for White British pupils (534), although it had increased by 15 scale points since 
2015. The average scores for other ethnic groups were not significantly different from the 
average score for White British pupils. Scores had also increased on average for Asian 
pupils (by 9 scale points) and pupils in the Other category (by 38 scale points). 


The average score for year 5 pupils whose first language was English (535) was not 
significantly different from the average score for pupils whose first language was not 
English (526). 


6.5 Does performance differ by gender, ethnicity and first 
language in year 9 science? 


Figure 50 below presents the performance of different pupil groups alongside the overall 
mathematics scores for 2019 and 2015. 


In 2019, the year 9 girls’ average score in science (518) was just above the boys’ 
average score (515); the difference was not significant. This mirrored the 2015 study 
when the difference between their average scores was also not significant (537 for girls 
and 536 for boys). 


In the highest-performing countries, the only significant differences between the 
performances of boys and girls were in Japan and the Republic of Korea, where boys 
performed significantly above girls. There were no significant differences in the other 
English-speaking countries, while in the European comparator countries the only 
significant differences were in Finland and Sweden, where girls performed higher than 
boys, and in Italy where the reverse was the case. Across year groups and subjects, 
these findings for Finland and Sweden were the only instances where girls performed 
significantly higher than boys. 


The average score for White British pupils (514) was significantly above the average 
score for Black pupils (481), but for all other ethnic groups average scores were not 
significantly different from the average score for White British pupils. In line with overall 
performance in science being significantly lower in 2019, the performance of pupils in all 
ethnic groups was lower than in 2015. 


Pupils whose first language was English achieved an average score of 515, which was 
significantly above the average score of 490 for pupils whose first language was not 
English. This 25 scale-point difference was higher than the difference in 2015 (13 scale 
points). 
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Figure 50: Average achievement in year 9 science by gender, ethnicity and first 
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Source: NPD and TIMSS 2019. 


Note 1: The average scores for Chinese pupils as a separate category were not included 
in the 2019 report due to the sample size being too small for the analysis to be robust. 


6.6 Does performance differ by socio-economic status? 


As set out in the chapter introduction, while pupils’ past and current eligibility for free 


school meals (FSM) is used as measure of disadvantage in England, TIMSS uses books 


at home as an internationally comparable proxy measure. 


Table 19 below shows that there was a good correlation between FSM eligibility and the 
TIMSS books-at-home measure in TIMSS 2019. Approximately double the percentage of 
year 5 and year 9 pupils who were eligible for FSM in the previous 6 years reported 10 or 
fewer books at home compared to pupils who had not been eligible for FSM. Fewer than 


half the percentage of year 5 and year 9 pupils who were eligible for FSM in the previous 
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6 years reported more than 200 books at home compared to pupils who had not been 


eligible for FSM. 


Table 20: The number of books at home and free school meal (FSM) eligibility 


Number of Percentage of | Percentage of | Percentage | Percentage of 
books year 5 pupils year 5 pupils of year 9 year 9 pupils 
eligible for not eligible for pupils not eligible for 
FSM FSM eligible for FSM 
FSM 
0-10 19.6% 7.8% 30.1% 15.7% 
11-25 29.3% 19.5% 28.9% 23.1% 
26-100 25.2% 34.9% 20.2% 27.3% 
101-200 10.0% 16.0% 7.8% 14.9% 
More than 200 5.5% 14.6% 5.0% 13.2% 
Unknown 10.5% 7.2% 8.0% 5.9% 


Source: NPD 2019. 


Note 1: Column percentages total 100%. 
Note 2: FSM eligibility refers to pupils eligible for free school meals at any point in the 
previous 6 years 


6.6.1 Did year 5 pupils’ performance in mathematics differ by FSM 
status and the number of books at home? 


Year 5 pupils who were eligible for FSM at any time in the previous 6 years achieved an 
average score of 525 in mathematics, significantly lower than the average score of 561 
for pupils who were not eligible for FSM. 


As shown in Figure 51 below, for year 5 pupils in England, increases in the number of 
books at home were positively and significantly associated with higher average 
mathematics scores, except in 1 instance. The average score for pupils with more than 
200 books at home was not significantly higher than the average score for pupils with 
between 101—200 books at home. 
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Figure 51: Average achievement in year 5 mathematics by 
eligibility for free school meals and number of books at home (England) 
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Source: NPD and TIMSS 2019. 
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6.6.2 Did year 9 pupils’ performance in mathematics differ by FSM 
status and the number of books at home? 


Year 9 pupils who were eligible for FSM at any time in the previous 6 years achieved an 
average mathematics score of 476 in 2019, significantly below the average score for 
pupils who were not eligible for FSM (522). 


For year 9 pupils in England, increases in the number of books at home were positively 
and significantly associated with average mathematics scores (see Figure 52 below). 


Figure 52: Average achievement in year 9 mathematics by 
eligibility for free school meals and number of books at home (England) 
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Source: NPD and TIMSS 2019. 
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6.6.3 Did year 5 pupils’ performance in science differ by FSM status 
and the number of books at home? 


Year 5 pupils who were eligible for FSM at any time in the previous 6 years achieved an 
average score of 510 in 2019 (see Figure 53 below), significantly below the average 
score of 542 for pupils who were not eligible for FSM. 


As in mathematics, for year 5 pupils in England, increases in the number of books at 
home were positively and significantly associated with average science scores in 2019, 
except in 1 instance. The average score for pupils with more than 200 books at home 
(573) was not significantly higher than the average score for pupils with between 101— 
200 books at home (566). 


Figure 53: Average achievement in year 5 science by 
eligibility for free school meals and number of books at home (England) 
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6.6.4 Did year 9 pupils’ performance in science differ by FSM status 
and the number of books at home? 


Pupils who were eligible for FSM at any time in the previous 6 years achieved an 
average score of 476, significantly below the average score of 523 for pupils who were 
not eligible for FSM. 


As shown in Figure 54 below, for year 9 pupils in England, increases in the number of 


books were associated with significantly higher average science scores. 


Figure 54: Average achievement in year 9 science by 
eligibility for free school meals and number of books at home (England) 
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6.7 What percentage of pupils reached each international 
benchmark by pupil characteristics? 


6.7.1 What percentage of pupils reached each international benchmark 
by pupil characteristics in year 5 mathematics? 


The percentages of year 5 boys reaching the advanced benchmark and high benchmark 
or above were significantly higher than the percentages of year 5 girls. However, as 
Figure 55 shows, at the intermediate and low benchmarks or above, the percentages 
were the same across the genders. 


There were no significant differences in the percentage of pupils who reached each of 
the benchmarks by ethnic group compared to White British pupils, except in 1 instance. 
Significantly more pupils from the Other ethnic group reached the low benchmark or 
above compared with White British pupils (99% compared to 96%). 


There were no significant differences in the percentage of year 5 pupils reaching each of 
the benchmarks by first language. 
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Figure 55: The percentage of year 5 pupils reaching the international benchmarks 
in mathematics by gender, ethnicity and first language (England) 
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Source: NPD and TIMSS 2019. 


Note 1: Chinese pupils were not included in the 2019 report due to the sample size being 
too small for the analysis to be robust. 


In 2019, the percentages of year 5 pupils reaching each of the benchmarks in 
mathematics were significantly smaller for pupils who had been eligible for FSM in the 
previous 6 years than for their non-FSM-eligible peers (see Figure 56 below). 


The percentage of pupils reaching each benchmark increased for each successive 
category of number of books, except for pupils who had more than 200 books at home 
where the percentages reaching the high, intermediate and low benchmarks or above 
were similar to those for pupils who had 101-200 books at home. However, 8 times the 
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percentage of pupils reporting more than 200 books at home reached the advanced 
benchmark compared with pupils reporting fewer than 10 books at home. 


Figure 56: The percentage of year 5 pupils reaching the international benchmarks 
in mathematics by eligibility for free school meals and number of books at home 
(England) 
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Source: NPD and TIMSS 2019. 


6.7.2 What percentage of pupils reached each international benchmark 
by pupil characteristics in year 9 mathematics? 


Figure 57 below shows a similar percentage of year 9 boys and girls reached all the 
benchmarks. 
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There were no significant differences in the percentage of pupils who reached each of 
the benchmarks by ethnic group in comparison with White British pupils. Similarly, there 
were no significant differences in the percentage of pupils achieving each of the TIMSS 
benchmarks by first language. 


Figure 57: The percentage of year 9 pupils reaching the international benchmarks 
in mathematics by gender, ethnicity and first language (England) 
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Source: NPD and TIMSS 2019. 


Note 1: Chinese pupils were not included in the 2019 report due to the sample size being 
too small for the analysis to be robust. 


In 2019, the percentages of year 9 pupils reaching each of the benchmarks in 


mathematics were significantly smaller for pupils who had been eligible for FSM in the 
previous 6 years than for their non-FSM eligible peers (see Figure 58 below). 
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The percentage of pupils reaching each benchmark increased for each successive 
category of number of books, except for pupils who had more than 200 books at home 
where the percentages reaching the intermediate and low benchmarks or above were 


similar to those for pupils with between 101 and 200 books at home. However, 


approximately 9 times the percentage of pupils reporting more than 200 books at home 
reached the advanced benchmark compared with pupils reporting fewer than 10 books at 


home. 


Figure 58: The percentage of year 9 pupils reaching the international benchmarks 
in mathematics by eligibility for free school meals and number of books at home 
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Source: NPD and TIMSS 2019. 
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6.7.3 What percentage of pupils reached each international benchmark 
by pupil characteristics in year 5 science? 


There were no significant differences between the percentages of year 5 boys and girls 
reaching each of the TIMSS benchmarks in science (see Figure 59 below). 


Significantly fewer Asian pupils reached the advanced benchmark compared with White 
British pupils (5% compared to 9%). There were no other significant differences between 
White British pupils’ performance and that of their peers from other ethnic groups. There 
were no significant differences by first language in the percentage of pupils achieving 
each of the TIMSS benchmarks. 


Figure 59: The percentage of year 5 pupils reaching the international benchmarks 
in science by gender, ethnicity and first language (England) 
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Source: NPD and TIMSS 2019. 
Note 1: Chinese pupils were not included in the 2019 report due to the sample size being 
too small for the analysis to be robust. 
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A significantly smaller percentage of year 5 pupils who were eligible for FSM at any time 
in the previous 6 years reached each of the TIMSS international benchmarks in science 
than pupils not eligible for FSM (see Figure 60 below). The percentage of pupils reaching 
each benchmark increased for each successive category of number of books, except for 
pupils who had more than 200 books at home, where the percentages reaching the 
intermediate and low benchmarks or above were similar to the percentage of pupils with 
101-200 books at home. More than 20 times the percentage of pupils reporting more 


than 200 books at home reached the advanced benchmark compared with pupils 


reporting fewer than 10 books at home. 


Figure 60: The percentage of year 5 pupils reaching the international benchmarks 
in science by eligibility for free school meals and number of books at home 
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Source: NPD and TIMSS 2019. 
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6.7.4 What percentage of pupils reached each international benchmark 
by pupil characteristics in year 9 science? 


There were no significant differences between the percentages of year 9 boys and girls 
reaching each of the TIMSS benchmarks in science (see Figure 61 below). 


Significantly fewer Black pupils reached the low benchmark or above compared with 
White British pupils (77% compared with 89%). There were no other significant 
differences in the percentages of pupils who achieved each of the benchmarks by ethnic 
group relative to White British pupils. Significantly fewer pupils whose first language was 
not English reached the intermediate and high TIMSS international benchmarks or 
above, but differences at the low benchmark or above and the advanced benchmark 
were not significant. 


Figure 61: The percentage of year 9 pupils reaching the international benchmarks 
in science by gender, ethnicity and first language (England) 
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Source: NPD and TIMSS 2019. 
Note 1: Chinese pupils were not included in the 2019 report due to the sample size being 
too small for the analysis to be robust. 
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A significantly lower percentage of year 9 pupils who were eligible for FSM at any time in 
the previous 6 years reached each of the TIMSS international benchmarks in science 
compared with pupils not eligible for FSM (see Figure 62 below). 


The percentage of pupils reaching each benchmark increased for each successive 
category of number of books, except for pupils who had more than 200 books at home, 
where the percentages reaching the intermediate and low benchmarks or above were 
similar to pupils with 101-200 books at home. However, 16 times the percentage of 
pupils reporting more than 200 books at home reached the advanced benchmark 
compared with pupils reporting fewer than 10 books at home. 


Figure 62: The percentage of year 9 pupils reaching the international benchmarks 
in science by eligibility for free school meals and number of books at home 
(England) 
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Chapter 7. Pupil attitudes and aspirations in 
mathematics and science 


This chapter summarises findings from the questionnaire on pupils’ attitudes towards 
mathematics and science and their aspirations in these subjects. Pupil questionnaire 
response rates in England were over 75%, which means that the data are robust for 

carrying out analysis for this chapter. 


The chapter’s sections focus on whether pupils in each subject: 


reported that lessons provide instructional clarity 
were confident in their abilities 

valued the subject”° 

liked learning it 

had aspirations to study it after age 16” 


Where there were interesting comparisons to be drawn between pupils in England and 
their peers in other comparator group countries, these are discussed. However, there can 
be difficulties when comparing pupil attitudes between different countries and cultures, 
for example varied expectations and experiences. 


The comparator countries referred to in this chapter are listed in section 1.5’. Only year 
9 pupils answered questions concerning valuing the subject. 


The chapter also describes whether or not these attitudinal factors were associated with 
higher or lower performance in the TIMSS assessments. However, it is important to note 
that an association (or correlation) between 2 variables (such as level of engagement 
and average achievement) is not the same as causation (i.e. that one thing causes the 
other). 


7.1 Main findings 


e Overall, analysis indicates that pupil confidence and, to a lesser extent, a liking for 
learning a subject were more strongly associated with achievement compared to 
instructional clarity and valuing the subject. 

e For pupils in year 9, instructional clarity and valuing the subject were more clearly 
associated with achievement in science than in mathematics. 

e There was a positive and significant association between England’s year 5 and 
year 9 pupils’ mathematics achievement and the extent to which they reported that 
their lessons provided instructional clarity. The same was true for year 9 science. 
There was no significant relationship between year 5 pupils in England’s science 


70 In all participating countries only pupils in year 9 were asked about the extent to which they valued the 
subject. 

™ This was an additional question posed for England’s pupils only on behalf of the Department for 
Education as part of the TIMSS questionnaires. 

72 Canada, Ireland, Germany, the Netherlands, Poland and Spain did not participate in the year 9 

study. Russia, Finland, France, Lithuania and Sweden did not participate in all the year 9 science 
questionnaires discussed in this chapter. Where they did participate, they are included in comparisons. 
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achievement and the extent to which they reported that their lessons provided 
instructional clarity. 

e There was a positive and significant association between confidence in 
mathematical ability and average achievement in both years 5 and 9. Pupils in 
England, in both year groups, who were very confident in mathematics scored 
over 100 scale points higher, on average, compared to their peers who were not 
confident. The same associations were evident in year 5 and year 9 science, 
although here the scale point differences were not as high (43 and 93). 

e Year 9 pupils who strongly valued mathematics and science respectively scored 
higher in each subject, on average, compared to their peers who did not value 
mathematics and science. 

e Both year 5 and year 9 pupils who very much liked learning mathematics scored 
higher, on average, compared to their peers who did not like learning 
mathematics. The same was evident in year 5 and year 9 science. 

e There was a positive and significant association between year 5 and 9 pupils’ 
higher average scores and the extent to which they agreed that they would like to 
study mathematics after secondary school, while for science this was evident for 
year 9 pupils only. 

e Across both year groups and both subjects, boys were more confident and more 
likely to report liking learning than girls were. Boys also valued both subjects more 
in year 9 (only year 9 pupils were asked how much they valued the subjects). 


7.2 To what extent did pupils in England report that their 
lessons provide instructional clarity in mathematics and 
science? 


Instructional clarity was a new focus for TIMSS 2019; previously pupils completed 
questionnaires on the extent to which they viewed their teaching to be engaging. 
Consequently, there are no comparisons to be made between 2019 data and previous 
TIMSS cycles. 


For both mathematics and science, pupils responded to the following statements using a 
4-point scale from ‘Agree a lot’ to ‘Disagree a lot’. Statements were consistent across 
both subjects, but statement 6 was used for year 9 pupils only. 


1) | know what my teacher expects me to do 

2) My teacher is easy to understand 

3) My teacher has clear answers to my questions 

4) My teacher is good at explaining mathematics/science 

5) My teacher does a variety of things to help us learn 

6) My teacher links new lessons to what | already know (year 9 only statement) 
7) My teacher explains a topic again when we don’t understand 


Based on their responses, scores were calculated that assigned pupils into 1 of 3 
categories related to the extent to which they reported that their lessons provided 
instructional clarity: high clarity, moderate clarity or low clarity’’. 


73 For full methodological explanations see the TIMSS 2019 International Report. 
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7.2.1 To what extent did pupils in England report that their lessons 
provided instructional clarity in mathematics? 


Year 5 


For year 5 pupils in England, there was a positive and significant association between the 
reporting of more instructional clarity and higher average mathematics achievement. 
Pupils who reported that their lessons provided high instructional clarity had significantly 
higher average achievement than those who reported moderate or low instructional 
clarity, while pupils who reported moderate instructional clarity had significantly higher 
average achievement than those who reported low instructional clarity. The difference 
between the average scores of those who reported high instructional clarity (561) and 
those who reported low instructional clarity (524) was 37 scale points, below the 
international average (42). 


The percentages of year 5 pupils in England who reported that their lessons provided 
high, moderate and low instructional clarity in mathematics were similar to the 
international averages (see Figure 63 below). 


Figure 63: The percentage of year 5 pupils reporting high, moderate and low 
instructional clarity in their mathematics lessons and their average achievement 
(England and international average) 
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Source: TIMSS 2019. 


More year 5 pupils in England reported that their lessons provided high instructional 
clarity compared to their peers from all of the highest-performing group of countries. 
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In comparison to the other English-speaking countries, more year 5 pupils in England 
reported that their mathematics lessons provided high instructional clarity compared to 
their peers in Australia, Canada and Northern Ireland, while the reverse was the case for 
Ireland, New Zealand and the United States. More pupils in England reported that their 
mathematics lessons provided high instructional clarity compared to their peers in any of 
the European comparator countries except for Lithuania and Spain. 


Year 9 


For year 9 pupils in England, there was also a positive and significant association 
between the reporting of more instructional clarity and higher average mathematics 
achievement. Pupils who reported that their lessons provided high instructional clarity 
had significantly higher average achievement than those who reported moderate or low 
instructional clarity. However, the difference in average achievement between pupils who 
reported moderate clarity and those who reported low instructional clarity was non- 
significant. As shown in Figure 64 below, the difference between the average scores of 
those who reported high instructional clarity (528) and those who reported low 
instructional clarity (507) was 21 scale points, below the international average (37). 


The percentages of year 9 pupils in England who reported that their lessons provided 
high, medium and low clarity of instruction were similar to the international averages (see 
Figure 64 below). 


Figure 64: The percentage of year 9 pupils reporting high, moderate and low 
instructional clarity in their mathematics lessons and their average achievement 
(England and international average) 
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Source: TIMSS 2019. 
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More year 9 pupils in England reported that their mathematics lessons provided high 
instructional clarity compared to their peers from all of the highest-performing group of 
countries. More year 9 pupils in England reported high instructional clarity compared to 
their peers in all 6 European comparator countries and all other English-speaking 
countries, except in the United States. 


7.2.2 To what extent did pupils in England report that their lessons 
provided instructional clarity in science? 


Year 5 


For year 5 pupils in England, the average science achievement for pupils who reported 
that their lessons provided high, moderate and low instructional clarity was not 
significantly different. As shown in Figure 65 below, the average score of those who 
reported high instructional clarity (539) was 1 point lower than those who reported low 
instructional clarity (540). In contrast, across all countries, pupils who reported high 
instructional clarity had average scores 32 points higher than those who reported low 
instructional clarity. 


The percentages of year 5 pupils in England who reported that their lessons provided 
high, medium or low clarity of instruction were similar to the international averages. 


Figure 65: The percentage of year 5 pupils reporting the extent of instructional 
clarity in their science lessons and their average achievement (England and 
international average) 


Percentage of Average 
pupils achievement 
| - 600 
539 538 540 
80 - - 500 
498 
480 466 
+ 400 
60 - 
+ 300 
40 - 
72 
+ 200 
20 - 
+ 100 
22 
mm: 8 
0; 1 0 
High clarity Moderate clarity Low clarity 
= England International Average 


Source: TIMSS 2019. 
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More year 5 pupils in England reported that their science lessons provided high 
instructional clarity compared to their peers in all the highest-performing countries except 
Russia. 


More year 5 pupils in England reported that their science lessons provided high 
instructional clarity compared to their peers in Australia and New Zealand, with the 
reverse being the case for the remaining 4 English-speaking countries. More pupils in 
England reported high instructional clarity in their science lessons than pupils in Finland, 
France, the Netherlands, Poland and Sweden. The position was reversed in the 
remaining 5 European comparator countries, where higher percentages of pupils 
reported high instructional clarity in their year 5 science lessons. 


Year 9 


For year 9 pupils in England, there was a positive and significant association between the 
reporting of more instructional clarity and higher average science achievement (see 
Figure 66 below). Pupils who reported that their lessons provided high instructional clarity 
had significantly higher average achievement than those who reported moderate or low 
instructional clarity, while pupils who reported moderate instructional clarity had 
significantly higher average achievement than those who reported low instructional 
clarity. The difference between the average scores of those who reported high 
instructional clarity (534) and those who reported low instructional clarity (487) was 47 
scale points, above the international average (41). 


The percentage of year 9 pupils in England who reported that their lessons provided high 
instructional clarity was below the international average, while the percentage of pupils 
who reported that their lessons provided low instructional clarity was above the 
international average. 


Across all 4 sets of analysis (mathematics and science in years 5 and 9), gender 
differences were significant only for year 9 science. A higher percentage of year 9 boys 
reported that their lessons provided high instructional clarity (44%) compared to girls 
(40%). Similarly, a higher percentage of year 9 girls reported that their lessons provided 
low instructional clarity (20%) compared to boys (14%). 
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Figure 66: The percentage of year 9 pupils reporting the extent of instructional 
clarity in their science lessons and their average achievement (England and 
international average) 
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Source: TIMSS 2019. 


More year 9 pupils in England reported that their science lessons provided high 
instructional clarity compared to their peers in all the highest-performing countries where 
this question was asked”. 


More pupils in England reported that their lessons provided high instructional clarity 
compared to their peers in Australia and Ireland. The reverse was the case compared 
with pupils in New Zealand and the United States. The same percentage of year 9 pupils 
in England reported that their lessons provided high instructional clarity as those in Italy, 
but this was reported by fewer pupils in Norway. 


In both mathematics and science, higher percentages of year 5 pupils reported high 
levels of instructional clarity compared to year 9 pupils (31 percentage points higher in 
mathematics and 28 percentage points higher in science). 


74 Russia did not ask this question but did ask an optional set of more detailed questions on each science 
domain. See the TIMSS 2019 International Report. 
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7.3 To what extent were pupils in England confident about 
their mathematics and science abilities? 


For both mathematics and science, pupils responded to the following statements using a 
4-point scale from ‘Agree a lot’ to ‘Disagree a lot’. These were consistent across both 
subjects except for statement 5, which was used in mathematics only. There were some 
variations in the number of statements and their phrasing in 2015, however these have 
not affected the IEA’s use of comparisons across TIMSS cycles. 


1) | usually do well in mathematics/science 

2) Mathematics/science is more difficult for me than for many of my classmates 
3) Mathematics/science is not one of my strengths 

4) | learn things quickly in mathematics/science 

5) Mathematics makes me nervous (no equivalent science statement) 

6) | am good at working out difficult mathematics/science problems 

7) My teacher tells me | am good at mathematics/science 

8) Mathematics/science is harder for me than any other subject 

9) Mathematics/science makes me confused 


Based on their responses, scores were calculated that assigned pupils into 1 of 3 
categories. These related to the extent to which they were confident in mathematics or 
science: 


e very confident 
e somewhat confident 
e not confident’® 


In 2015, different pupil category titles were used in some cases. For example, ‘confident’ 

was used as the middle category for reporting pupil confidence in 2015 while ‘somewhat 

confident’ was used for this in 2019. When reporting differences between 2015 and 2019, 
the latest category title is used. 


7.3.1 To what extent were pupils in England confident about their 
mathematical abilities? 


Year 5 


As in 2011 and 2015, there was a positive and significant association between having 
more confidence in mathematical ability and higher average achievement in 2019. Year 5 
pupils in England who reported that they were very confident in their mathematical ability 
had significantly higher average achievement than those who reported that they were 
somewhat or not confident in their mathematical ability, while pupils who reported that 
they were somewhat confident in their mathematical ability had significantly higher 
average achievement than those who reported that they were not confident. The 
difference between the average score of those who reported that they were very 
confident (607) and those who reported that they were not confident (506) was 101 scale 
points, above the international average (89) (see Figure 67 below). 


75 For full methodological explanations see the TIMSS 2019 International Report. 
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In comparison with 2015, the average achievement of year 5 pupils in each category was 
higher in 2019, in line with England’s higher overall average score. However, average 
achievement for very confident pupils has increased more than for pupils in the other 
categories, increasing by nearly 30 scale points in 2019 (607 in 2019 compared with 578 
in 2015). This has widened the range of average achievement scores between the 
highest and lowest categories (from 79 to 101). 


The percentages of England’s pupils who were very confident or somewhat confident 
were similar to the international averages in 2019. However, the percentage of pupils in 
England who were either very confident or somewhat confident has decreased over time: 
81% in 2011, 80% in 2015 and 76% in 2019. 


The percentage of pupils in England who were not confident was similar to the 
international average (24% compared with 23%). This 24% was above England’s 
percentages for both 2015 and 2011 (20% and 19% respectively), corresponding with the 
decrease over time in the percentage of year 5 pupils who were very or somewhat 
confident in mathematics, described above. 


There were some significant gender differences in year 5 pupils’ reported confidence in 
mathematics. Significantly more boys were very confident in comparison to girls (39% 
compared to 24%), while, similarly, significantly more girls were not confident (30% 
compared to 18%). 


Figure 67: The percentage of year 5 pupils reporting the extent to which they were 
confident in their mathematical ability and their average achievement (England and 
international average) 
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Source: TIMSS 2019. 
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More year 5 pupils in England were very confident in mathematics compared to their 
peers in all of the highest-performing countries. The percentage of pupils in England who 
were very confident was similar to the majority of English-speaking countries — all but 
New Zealand (20%) were clustered within a 4 percentage-point range from Australia’s 
29% to Ireland’s 33%, with England at 31%. The percentage of very confident pupils in 
England was lower than in most of the European comparator countries. Only pupils in 
Lithuania, Spain and Poland had lower percentages compared to their peers in England. 


Year 9 


In 2019, as in 2011 and 2015, there was a positive and significant association between 
having more confidence in mathematical ability and higher average achievement in year 
9 mathematics. Year 9 pupils in England who reported that they were very confident in 
their mathematical ability had significantly higher average achievement than those who 
reported that they were somewhat or not confident in their mathematical ability, while 
pupils who reported that they were somewhat confident in their mathematical ability had 
significantly higher average achievement than those who reported that they were not 
confident. The difference between the average scores of those who reported that they 
were very confident (588) and those who reported that they were not confident (480) was 
108 scale points, in line with the international average (106) (see Figure 68 below). 


As with year 5 pupils, there was an increase in the average achievement of very 
confident pupils in 2019 (588) compared with 2015 (578), although the overall 
mathematics score for year 9 pupils in 2019 was similar to 2015. This has widened the 
range of average achievement scores between these categories (from 99 scale points in 
2015 to 108 scale points in 2019). The percentage of year 9 pupils in England who were 
very confident in mathematics was similar to the international average, while fewer pupils 
were not confident in mathematics compared to the international average (38% 
compared to 44%). 


The percentage of England’s pupils who were either very confident or somewhat 
confident has decreased over time: 69% in 2011, 65% in 2015 and 63% in 2019. The 
38% of pupils who were not confident was above England’s percentages in both 2015 
and 2011 (35% and 32% respectively), corresponding with the downward trend in the 
percentage of very or somewhat confident pupils in year 9 mathematics over the same 
period. 


As in year 5, there were some significant gender differences in year 9 pupils’ reported 
confidence in mathematics. Significantly more boys were very confident in comparison to 
girls (18% compared to 10%), while, similarly, significantly more girls were not confident 
(44% compared to 30%). 
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Figure 68: The percentage of year 9 pupils reporting the extent to which they were 
confident in their mathematical ability and their average achievement (England and 
international average) 
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Source: TIMSS 2019. 


As in year 5, more of England’s year 9 pupils were very confident in mathematics 
compared to their peers in any of the highest-performing countries. In comparison to the 
other English-speaking countries, more pupils in England than in New Zealand were very 
confident in mathematics, but fewer pupils in England were when compared with those in 
Ireland and the United States. Australia’s percentage was the same as England’s. As in 
year 5, more pupils were very confident in mathematics in most comparator European 
countries, with only Lithuania and France having lower percentages of very confident 
pupils compared to England. 


7.3.2 To what extent were pupils in England confident about their 
science abilities? 


Year 5 


As in 2011 and 2015, there was a positive and significant association between having 
more confidence in science ability and higher average year 5 science achievement in 
2019. Year 5 pupils in England who reported that they were very confident in their 
science ability had significantly higher average achievement than those who reported that 
they were somewhat or not confident in their science ability, while pupils who reported 
that they were somewhat confident in their science ability had significantly higher 
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average achievement than those who reported that they were not confident. The 
difference between the average score of those who reported that they were very 
confident (559) and those who reported that they were not confident (516) was 43 scale 
points, below the international average (67) (see Figure 69 below). This scale point 
difference was smaller than in 2015 (46 scale points) and the same as in 2011. 


As in 2015, the average achievement of year 5 pupils in 2019 was higher for those in the 
very confident and not confident categories and similar in the somewhat confident 
category. The percentage of England’s year 5 pupils who were either very confident or 
somewhat confident in science was the same as in 2015 (75%). 

The percentage of pupils who were very confident in science was below the international 
average (28% compared to 38%). This 28% was also lower than England’s percentages 
in 2015 and 2011 (both 33%). More year 5 pupils in England were not confident 
compared to the international average (25% compared to 19%). This outcome was the 
same as for England in 2015, but a reduction compared to 2011 (29%). Year 5 pupils 
performed above the international average in each confidence level but the difference 
was largest for pupils who were not confident (63 scale points). 


Figure 69: The percentage of year 5 pupils reporting the extent to which they were 
confident in their science ability and their average achievement (England and 
international average) 


Percentage of Average 
pupils achievement 
559 - 600 
538 
oo 
80 - 520 ~ 500 
486 
453 
+ 400 
60 - 
- 300 
40 - 
+ 200 
20 - 43 
2g ~ 400 
19 
0 1 0 
Very confident Somewhat confident Not confident 
= England International Average 


Source: TIMSS 2019. 
A higher percentage of England’s year 5 pupils were very confident in science than was 


the case for their peers in the highest-performing countries, except for Chinese Taipei 
and Russia. 
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Fewer year 5 pupils in England were very confident in science compared to their peers in 
any of the other English-speaking countries, except New Zealand (21%). Similarly, fewer 
pupils in England were very confident compared to their peers in most European 
comparator countries. Only Finland and France had lower percentages of very confident 
pupils (both were 1 percentage point lower). 


Year 9 


As in 2011 and 2015, there was a positive and significant association between having 
more confidence in science ability and higher average year 9 science achievement in 
2019. Year 9 pupils in England who reported that they were very confident in their 
science ability had significantly higher average achievement than those who reported that 
they were somewhat or not confident, while pupils who reported that they were 
somewhat confident had significantly higher average achievement than those who 
reported that they were not confident. The difference between the average scores of 
those who reported that they were very confident (581) and not confident (488) was 93 
scale points, in line with the international average (91) (see Figure 70 below). This has 
widened the range of average achievement scores between these categories (from 82 
scale points in 2015 to 93 scale points in 2019). 


There was a small decrease in the average achievement of very confident pupils in 2019 
(581) compared with 2015 (585). The percentage of year 9 pupils who were very 
confident in science was below the international average (15% compared to 23%) 
whereas in 2015 it was similar (21% for England compared to 22%). 


The percentage of England’s pupils who were either very confident or somewhat 
confident has decreased over time: 75% in 2011, 62% in 2015 and 53% in 2019. Nearly 
half of England’s pupils considered themselves not confident in science in 2019, which 
was above the international average (48% compared to 38%). This 48% was higher than 
England’s percentages for both 2015 and 2011 (38% and 25% respectively), 
corresponding with the downward trend in the percentage of very or somewhat confident 
pupils in year 9 science over the same period. 


Although there were no significant gender differences in year 5 pupils’ reported 
confidence in science, significant gender differences did exist for year 9 pupils. 
Significantly more boys than girls were very confident (18% compared to 12%), while, 
similarly, significantly more girls were not confident (52% compared to 42%). 


Across mathematics and science in England, the percentages of pupils who were not 


confident increased between years 5 and 9, while correspondingly the percentages who 
were very confident approximately halved. 
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Figure 70: The percentage of year 9 pupils reporting the extent to which they were 
confident in their science ability and their average achievement (England and 
international average) 
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Source: TIMSS 2019. 


More year 9 pupils in England were very confident in science compared to their peers in 
all of the highest-performing countries, except Singapore. Fewer pupils in England were 
very confident than was the case for their peers in any of the other English-speaking 
countries, except New Zealand. Similarly, fewer pupils in England were very confident 
than pupils in Italy and Norway (the only 2 European comparator countries that used the 
questionnaire survey). 


7.4 To what extent did year 9 pupils in England value 
mathematics and science? 


For both mathematics and science, only year 9 pupils responded to the following 
statements using a 4-point scale from ‘Agree a lot’ to ‘Disagree a lot’. All statements were 
common to both subjects with just the subject title changed, and were the same as in 
2015. 


1) | think learning mathematics/science will help me in my daily life 

2) | need mathematics/science to learn other school subjects 

3) | need to do well in mathematics/science to get into the university of my choice 

4) | need to do well in mathematics/science to get the job | want 

5) | would like a job that involves using mathematics/science 

6) It is important to learn about mathematics/science to get ahead in the world 

7) Learning mathematics/science will give me more job opportunities when | am an adult 
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8) My parents think that it is important that | do well in mathematics/science 
9) It is important to do well in mathematics/science 


Based on their responses, scores were calculated that assigned pupils into 1 of 3 
categories. These related to the extent to which they valued the subject: 


e strongly value 
e somewhat value 
e donot value’ 


In 2015, different pupil category titles were used in some cases. For example, ‘value’ was 
used as the middle category for reporting in 2015 while ‘somewhat value’ was used for 
this in 2019. When reporting differences between 2015 and 2019, the latest category title 
is used. 


7.4.1 To what extent did year 9 pupils value mathematics? 


As in 2011 and 2015, there was a positive and significant association in 2019 between 
year 9 pupils in England valuing mathematics and average achievement. In 2019, pupils 
who strongly valued mathematics had significantly higher average achievement than 
those who somewhat valued or did not value mathematics, while pupils who somewhat 
valued mathematics had significantly higher average achievement than those who did not 
value mathematics. The difference between the average score of those who strongly 
valued mathematics (528) and those who did not value mathematics (500) was 28 scale 
points, below the international average (45) (see Figure 71 below). In 2019, the average 
achievement of year 9 pupils who strongly valued mathematics was approximately the 
same as it was in 2015, while achievement for pupils who did not value mathematics was 
10 scale points higher in 2019 than 2015 (500 compared with 490). 


The percentage of pupils who strongly valued mathematics was similar to the 
international average (38% compared to 37%). However, this 38% was lower than the 
percentage for England in both 2015 and 2011 (46% and 48% respectively). Year 9 
pupils performed above the international average in each level of valuing the subject, but 
the difference was largest for pupils who did not value mathematics (38 scale points). 


The percentage of year 9 pupils who either strongly valued or somewhat valued 
mathematics was similar in 2019 (89%) to 2015 (92%) and 2011 (91%). Fewer year 9 
pupils in England reported that they did not value mathematics compared to the 
international average (10% compared to 16%). This 10% was the same as England’s 
percentage in 2011 and similar to the percentage in 2015 (8%). 


Significantly more boys strongly valued the subject than girls (43% compared to 34%), 
while, similarly, significantly more girls did not value the subject (12% compared to 9%). 


76 For full methodological explanations see the TIMSS 2019 International Report. 
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Figure 71: The percentage of year 9 pupils reporting the extent to which they value 
mathematics and their average achievement (England and international average) 
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Source: TIMSS 2019. 


More year 9 pupils in England strongly valued mathematics than was the case for their 
peers from any of the highest-performing comparator group countries. In comparison to 
the English-speaking countries, more pupils in England strongly valued mathematics than 
pupils in Ireland and New Zealand, while fewer did than in the United States. In Australia, 
the same percentage of pupils strongly valued mathematics as in England. More year 9 
pupils in England strongly valued mathematics than was the case for their peers in all 6 
European comparator countries. 


7.4.2 To what extent did year 9 pupils value science? 


As in 2011 and 2015, there was a positive and significant association between year 9 
pupils in England valuing science and average achievement in 2019. Pupils who strongly 
valued science in 2019 had significantly higher average achievement than those who 
somewhat valued or did not value science, while pupils who somewhat valued science 
had significantly higher average achievement than those who did not value science. The 
difference between the average score of those who strongly valued science (540) and 
those who did not value science (491) was 49 scale points, above the international 
average (44) (see Figure 72 below). In comparison with 2015, the average achievement 
of year 9 pupils who strongly valued science was 18 scale points lower in 2019 (540 
compared with 558) and 11 scale points lower for those who did not value science (491 
compared with 502), in line with England’s overall lower performance in science in 2019. 
The percentage of England’s pupils who strongly valued science (33%) was 3 
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percentage points below the international average in 2019 (36%), and below the 
percentages for England in both 2015 and 2011 (39% and 41% respectively). 


The percentage of pupils who either strongly valued or somewhat valued science in 2019 
(78%) was lower than in 2015 (82%) but the same as in 2011. The percentage of year 9 
pupils in England who did not value science was the same as the international average 
(22%). This percentage was above England’s 2015 percentage (18%) but the same as in 
2011. 


Significantly more boys strongly valued the subject in comparison to girls (36% compared 
to 32%), while, similarly, significantly more girls did not value the subject (24% compared 
to 19%). 


Figure 72: The percentage of year 9 pupils reporting the extent to which they value 
science and their average achievement (England) 
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Source: TIMSS 2019. 


More year 9 pupils in England strongly valued science than was the case for their peers 
from any of the group of highest-performing countries, except Singapore. More year 9 
pupils in England strongly valued science than was the case for their peers in any of the 
English-speaking comparator countries, except the United States, as well as in the 6 
European comparator countries. 
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7.5 To what extent did pupils in England like learning 
mathematics and science? 


For both mathematics and science, pupils responded to the following statements using a 
4-point scale from ‘Agree a lot’ to Disagree a lot’. There were some common statements 
between subjects and some that were different, as noted below. The statements were 
consistent with those used in 2015. 


1) | enjoy learning mathematics/science 

2) | wish | did not have to study mathematics/science 

3) Mathematics/science is boring 

4) | learn many interesting things in mathematics/science 

5) | like mathematics/science 

6) | like any schoolwork that involves numbers/science teaches me how things in the 
world work 

7) | like to solve mathematics problems/lI like to conduct science experiments 

8) | look forward to mathematics lessons/I look forward to learning science 

9) Mathematics/science is one of my favourite subjects 


Based on their responses, scores were calculated that assigned pupils into 1 of 3 
categories related to the extent to which they liked learning: 


e very much like learning 
e somewhat like learning 
e do not like learning”’ 


In 2015, different pupil category titles were used in some cases. For example, ‘like 
learning’ was used as the middle category for reporting in 2015 while ‘somewhat like 
learning’ was used for this in 2019. When reporting differences between 2015 and 2019, 
the latest category title is used. 


7.5.1 To what extent did year 5 and year 9 pupils like learning 
mathematics? 


Year 5 


As in 2011 and 2015, there was a positive and significant association between liking 
mathematics and higher average mathematics achievement for year 5 pupils in 2019. 
Year 5 pupils in England who reported that they very much liked learning mathematics 
had significantly higher average achievement than those who reported that they 
somewhat liked or did not like learning mathematics, while pupils who reported that they 
somewhat liked learning mathematics had significantly higher average achievement than 
those who reported that they did not like learning mathematics. The difference between 
the average scores of those who reported that they very much liked learning 
mathematics (576) and those who reported that they did not like learning mathematics 
(530) was 46 scale points, above the international average (41) (see Figure 73 below). 
This 46 scale-point difference was larger than the range of average achievement scores 
between these categories in 2015 (32 scale points) and 2011 (18 scale points). 


™ For full methodological explanations see the TIMSS 2019 International Report. 
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In comparison with 2015, the average achievement of year 5 pupils in each category was 
higher in 2019, with pupils who very much liked learning mathematics scoring 21 scale 
points more (576 compared with 555), in line with England’s higher overall average 
score. 


The percentages of England’s year 5 pupils who very much liked learning mathematics 
and somewhat liked learning mathematics were similar to the international average. The 
percentage of pupils who either very much liked learning mathematics or somewhat liked 
learning mathematics in 2019 (78%) was below the equivalent percentages for 2015 
(82%) and 2011 (81%). More pupils in England did not like learning mathematics 
compared to the international average (23% compared to 20%). This 23% was an 
increase on England’s percentages for both 2015 and 2011 (17% and 19% respectively). 
Significantly more boys very much liked learning mathematics than girls (52% compared 
to 35%), while, similarly, significantly more girls did not like learning the subject (28% 
compared to 17%). 


Figure 73: The percentage of year 5 pupils reporting the extent to which they like 
learning mathematics and their average achievement (England and international 
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Source: TIMSS 2019. 


More year 5 pupils in England very much liked learning mathematics than was the case 
for their peers from all of highest-performing comparator group countries. Similarly, more 
year 5 pupils in England very much liked learning mathematics compared with their peers 
from any of the other English-speaking countries. More pupils in England very much liked 
learning mathematics than was the case for their peers in the European comparator 
group countries, with the exception of pupils in France, Italy and Lithuania. 
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Year 9 


As in 2011 and 2015, there was a positive and significant association between liking 
mathematics and higher average mathematics achievement for year 9 pupils in 2019. 
Year 9 pupils in England who reported that they very much liked learning mathematics 
had significantly higher average achievement than those who reported that they 
somewhat liked or did not like learning mathematics, while pupils who reported that they 
somewhat liked learning mathematics had significantly higher average achievement than 
those who reported that they did not like learning mathematics. The difference between 
the average scores of those who reported that they very much liked learning 
mathematics (552) and those who reported that they did not like learning mathematics 
(500) was 52 scale points, below the international average (62) (see Figure 74 below). 
This was lower than in 2015 when this difference was 60 scale points and 2011 when it 
was 64 scale points. 


In 2019, the average achievement of year 9 pupils who somewhat liked learning 
mathematics (530) and who did not like learning mathematics (500) was similar to 2015 
(532 and 499 respectively). However, average achievement was lower in 2019 for pupils 
who very much liked learning mathematics (552 compared with 559). 

While the percentage of England’s year 9 pupils who somewhat liked learning 
mathematics was similar to the international average, the percentage of pupils who very 
much liked learning mathematics was 8 percentage points below this (12% compared to 
20%). Consequently, the percentage of pupils who very much liked or somewhat liked 
learning mathematics was 50% in 2019, lower than in both 2015 (53%) and 2011 (58%). 
Year 9 pupils performed above the international average in each level of liking the 
subject, but the difference was largest for pupils who did not like learning mathematics 
(32 scale points). 


Half of England’s year 9 pupils did not like learning mathematics, above the international 
average (50% compared to 41%). This 50% was an increase on England’s percentages 
for both 2015 and 2011 (48% and 42% respectively). 


As in year 5, significantly more boys very much liked learning mathematics than girls 


(15% compared to 10%), while, similarly, significantly more girls did not like learning the 
subject (56% compared to 44%). 
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Figure 74: The percentage of year 9 pupils reporting the extent to which they like 
learning mathematics and their average achievement (England and the 
international average) 
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Source: TIMSS 2019. 


Compared with the countries in the highest-performing group, more year 9 pupils in 
England very much liked learning mathematics than was the case for their peers in 
Chinese Taipei, Japan and the Republic of Korea, while the reverse was the case for the 
remaining countries’ pupils. Fewer pupils in England very much liked learning 
mathematics than was the case for their peers in any of the 4 comparator English- 
speaking countries. Compared with the 6 European comparator countries, more year 9 
pupils in England very much liked learning mathematics than was the case for their peers 
in Finland and France, while England’s percentage was the same as the percentages for 
pupils in Lithuania and Norway. Fewer pupils in England very much liked learning 
mathematics than pupils in Italy and Sweden. 


7.5.2 To what extent did year 5 and year 9 pupils like learning science? 
Year 5 


As in 2011 and 2015, there was a positive and significant association between liking 
science and higher average science achievement for year 5 pupils in 2019. Year 5 pupils 
in England who reported that they very much and somewhat liked learning mathematics 
had significantly higher average achievement than those who reported that they did not 
like learning mathematics, however pupils who reported that they very much and 
somewhat liked learning mathematics had average achievement that was not 
significantly different. The difference between the average score of those who reported 
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that they very much liked learning mathematics (542) and those who reported that they 
did not like learning mathematics (528) was 14 scale points, below the international 
average (39) (see Figure 75 below). In 2015, this difference was 19 scale points and in 
2011, 17 scale points. In 2019, the average achievement of year 5 pupils who very much 
liked learning science was the same in 2015, while it was higher in 2019 by 5 scale 
points in both of the other 2 categories. 


The percentage of pupils who very much liked learning science was 6 percentage points 
below the international average (46% compared to 52%). This 46% is lower than the 49% 
of pupils who very much liked learning science in 2015 but higher than in 2011 (44%). 
More pupils in England did not like learning science compared to the international 
average (16% compared to 12%). However, this 16% was lower than for England in both 
2011 and 2015 (21% and 17% respectively). Year 9 pupils performed above the 
international average in each level of liking the subject but the difference was largest for 
pupils who did not like learning science (61 scale points). Across all countries as a whole 
the difference between pupils liking and not liking science and their average achievement 
was larger compared to pupils in England (a scale point difference of 39 compared to 
14). 


Significantly more year 5 boys than girls very much liked learning science (48% 
compared to 44%), while significantly more girls somewhat liked learning science (40% 
compared to 35%). 


Figure 75: The percentage of year 5 pupils reporting the extent to which they liked 
learning science and their average achievement (England and international 


average) 
Percentage of Average 
pupils achievement 
100 - 
+ 600 
6 540 528 
SS t—“S—( 
4 + 500 
sa 506 
a8 467 
+ 400 
60 - 
+ 300 
40 - 
+ 200 
52 
20 - 
36 - 100 
* 12 
0; 1 1 + 0 
Very much like Somewhat like Do not like 


= England International Average 


Source: TIMSS 2019. 
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In comparison to pupils from the highest-performing comparator countries, more year 5 
pupils in England very much liked learning science compared to their peers in the 
Republic of Korea and Russia. The reverse was the case in the remaining 4 countries. 
Fewer pupils in England very much liked learning science compared to their peers in all 
of the English-speaking countries. More pupils in England very much liked learning 
science than was the case for pupils in half of the European comparator group countries 
(Germany, Italy, Lithuania, Norway and Spain), while the reverse was the case for 
remaining half. 


Year 9 


As in 2011 and 2015, there was a positive and significant association between liking 
science and higher average science achievement for year 9 pupils in 2019. Year 9 pupils 
in England who reported that they very much liked learning science had significantly 
higher average achievement than those who reported that they somewhat liked or did not 
like learning science, while pupils who reported that they somewhat liked learning 
science had significantly higher average achievement than those who reported that they 
did not like learning science. The difference between the average scores of those who 
reported that they very much liked learning science (556) and those who reported that 
they did not like learning science (485) was 71 scale points, above the international 
average (64) (see Figure 76 below). In 2019, the average achievement of year 9 pupils 
who very much liked learning science (556) was 13 scale points lower than in 2015 (569). 
Similarly, in 2019, it was 19 scale points lower for pupils who did not like learning science 
than in 2015 (485 compared to 504). This has widened the range of average 
achievement scores between categories from 65 scale points in 2015 to 71 scale points 
in 2019. 


The percentage of pupils who very much liked learning science was 11 points below the 
international average (24% compared to 35%). This 24% was also below the 
percentages for England’s pupils in 2015 (32%) and 2011 (31%). More pupils in England 
did not like learning science compared to the international average (30% compared to 
20%). This 30% is higher than the percentages of pupils who did not like science in 2015 
(25%) and 2011 (23%), corresponding with the lower percentage of pupils who very 
much liked science in 2019 identified above. 


Significantly more boys very much liked learning science in comparison to girls (28% 


compared to 21%), while, similarly, significantly more girls did not like learning the 
subject (35% compared to 25%). 
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Figure 76: The percentage of year 9 pupils reporting the extent to which they liked 
learning science and their average achievement (England and international 


average) 
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Source: TIMSS 2019. 


More pupils in England very much liked learning science than was the case for their 
peers in all of the highest-performing comparator countries, except in Singapore. Fewer 
pupils in England very much liked learning science than their peers in any of the other 4 
English-speaking countries and in comparison with both of the European comparator 
countries, Italy and Norway. 


7.6 To what extent were the 4 pupil attitude factors associated 
with achievement? 


Figures 77 and 78 below compare the attitudinal factors from this chapter and the extent 
to which they were associated with pupils’ average achievement in England. This was 
achieved through comparing the average score for pupils in the highest and lowest 
categories to calculate a range (shown by the bars below). 


It should be noted that while associations between different factors and average 
achievement can be made, this does not mean the associations are causal. 


As in 2015, across all these attitudinal factors (instructional clarity, confidence in ability, 


valuing the subject and liking the subject), confidence was most strongly associated with 
average achievement. 
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As in 2015, this was most evident in year 9 mathematics (see Figures 15 and 16 below), 
in which the difference between the average score for pupils in England who were very 
confident compared with their not-confident peers was 108 scale points (in 2015 the 
difference was 99 scale points). This was more than 5 times the difference for 
instructional clarity (21) and more than double the difference for liking mathematics (52). 


Similarly, the difference related to confidence in mathematics for pupils in year 5 (101 
scale points, up from 79 scale points in 2015) was more than double the differences for 
the other 2 factors and also more than double the difference for confidence in science. 


The differences in mathematics average scores for pupils’ confidence in their ability was 
similar in years 5 and 9 (101 compared with 108). However, in year 9 science, the 
difference in scores between the most and least confident pupils was more than double 
the difference for year 5 science (94 in year 9 compared with 43 in year 5). 


The second largest differences related to liking learning the subject, as in 2015. The 
differences between the average scores for pupils who very much liked learning the 
subject compared with pupils who did not like learning the subject were greater in year 9 
than in year 5 for this factor. In mathematics, year 9 pupils’ average score difference for 
liking mathematics was just over 5 times greater than the difference for year 5 pupils (71 
compared with 14). 


In year 9, the difference between the average scores for pupils reporting high 
instructional clarity in science compared with those who reported low clarity was more 
than double that in mathematics (46 compared with 21), while it was nearly double for 
valuing the subject (49 compared with 28). Differences for instructional clarity were also 
evident when comparing year groups. In mathematics, instructional clarity had a greater 
association with year 5 pupils average scores (37) compared with year 9 pupils’ (21). 
However, in science, there was no association between instructional clarity and year 5 
pupils’ average scores (-1), but an association was present for year 9 pupils (46). 
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Figure 77: Differences in average achievement by pupil attitude in mathematics 
and science (England, Year 5) 
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Source: TIMSS 2019. 


Note 1: Pupils who reported high instructional clarity had lower average achievement 
than pupils who reported low clarity. 


Figure 78: Differences in average achievement by pupil attitude in mathematics 
and science (England, Year 9) 
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Source: TIMSS 2019. 


Overall, analysis of these factors indicates that pupil confidence and, to a lesser extent, a 
liking for learning the subject (and the statements related to these overarching categories 
in the TIMSS questionnaires) were more strongly associated with achievement compared 
to the other 2 factors. Analysis also indicates that instructional clarity and valuing the 
subject in year 9 science were more strongly associated, on average, with achievement 
than was the case for mathematics. 


As in 2015, the same findings apply across participating countries as a whole, although it 
is noted above that pupils in the 5 East Asian countries (all of which are among the 
highest-performing countries) generally had low scores in these 4 factors by international 
standards, highlighting the difficulties involved in comparing pupil attitudes between 
countries and cultures. 
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In Figures 79 and 80 below, the differences between year 5 girls’ and boys’ average 
scores in mathematics and science were calculated as above by taking the highest and 
lowest categories to calculate a range in each of the attitudinal areas. When tested for 
significance, there were no significant differences by gender in these ranges. 


Figure 79: Differences in average achievement in mathematics by pupil attitude 
and gender (England, Year 5) 
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Figure 80: Differences in average achievement in science by pupil attitude and 
gender (England, Year 5) 
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Note 1: Girls who reported high instructional clarity had lower average achievement than 
girls who reported low clarity. 


In Figures 81 and 82 below, the same differences between year 9 girls’ and boys’ 
average scores in mathematics and science were calculated for the highest and lowest 
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categories to calculate a range in each of the attitudinal areas. Differences in average 
scores in mathematics and science scores by confidence in ability were significantly 
different for boys and girls, but for the other attitudes there were no significant differences 
by gender. There was a greater difference in boys’ average achievement compared to 
girls’ average achievement in both mathematics and science related to confidence. 


Figure 81: Differences in average achievement in mathematics by pupil attitude 
and gender (England, Year 9) 
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Source: TIMSS 2019. 


Figure 82: Differences in average achievement in science by pupil attitude and 
gender (England, Year 9) 
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7.7 To what extent do pupils aspire to study mathematics and 
science after age 16? 


This was an additional question for England’s pupils only, posed on behalf of the 
Department for Education as part of the TIMSS questionnaires. 


As shown in Figure 83 below, there was a positive and significant association between 
year 5 and year 9 pupils’ average scores and the extent to which they agreed that they 
would like to study mathematics after secondary school. Pupils who strongly agreed that 
they would like to study mathematics had significantly higher average scores, while those 
who strongly disagreed had lower average scores. In year 5, nearly two-thirds of pupils 
either strongly agreed or agreed they would like to study mathematics after secondary 
school. However, in year 9, this proportion was half of all pupils, with the remaining half 
disagreeing or strongly disagreeing with this aspiration. 


Figure 83: The percentage of year 5 and year 9 pupils reporting agreement that 
they would like to study mathematics after secondary school and their average 
achievement in mathematics (England) 
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In science there was a positive and significant association between year 9 pupils’ 
average scores and the extent to which they agreed that they would like to study science 
after secondary school, but there was no association for year 5 pupils (see Figure 84 
below). In year 9, pupils who strongly agreed had significantly higher average scores 
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than those who strongly disagreed. In year 5, pupils who either agreed or disagreed had 
significantly higher average scores than pupils who strongly disagreed. However, pupils 
who strongly agreed had an average score that was not significantly different from the 
average score for pupils who agreed and that was significantly lower than the average 
score for pupils who disagreed. Just over half of year 5 pupils either strongly agreed or 
agreed that they would like to study science after secondary school, while in year 9, the 
proportion was exactly half. In both years, approximately a quarter of pupils strongly 
disagreed with the statement. 


Figure 84: The percentage of year 5 and year 9 pupils reporting agreement that 
they would like to study science after secondary school and their average 
achievement in science (England) 
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Source: TIMSS 2019. 
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Chapter 8. School environment and resources 


This chapter summarises findings from headteacher, teacher and pupil questionnaires on 
aspects of school environment and resources. 


Chapter sections below focus on the extent to which year 5 and year 9 pupils: 


e were taught in schools where headteachers reported an emphasis on academic 
SUCCESS 

e were taught in schools where headteachers reported instruction was affected by 

resource shortages 

were taught in schools where headteachers reported discipline problems 

were taught in schools that teachers reported were safe and orderly 

experienced bullying behaviours in school (pupil questionnaire) 

reported disorderly behaviour in school (pupil questionnaire). 


Using these findings, we discuss the noteworthy comparisons drawn between pupils in 
England and their peers in other comparator group countries. The comparator countries 
referred to in this chapter are listed in section 1.578. 


The chapter also describes whether or not these factors were associated with higher or 
lower performance in the TIMSS assessments, although it is important to note that an 
association (or correlation) between 2 variables (such as level of engagement and 
average achievement) is not the same as causation (i.e. that one thing causes the other). 


The response rates for the headteachers’ and teachers’ questionnaires in England were 
lower than in the majority of countries in TIMSS 2019. As a result, data on the extent to 
which pupils were taught in schools with an emphasis on academic success, the extent 
to which headteachers reported resource shortages, teachers’ perceptions of discipline 
problems and the safety and orderliness of the school were available for fewer than 70% 
of pupils in England, the threshold the IEA sets for its International Exhibits. The 
response rates were taken into account in the analysis for this section and findings were 
only included if they were based on data for more than 50% of pupils. Questions in this 
chapter based on the pupil questionnaire (experiences of bullying and disorderly 
behaviour in mathematics classes) were unaffected and had response rates of at least 
70%. 


78 Canada, Ireland, Germany, the Netherlands, Poland and Spain did not participate in the year 9 study. 
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8.1 Main findings 


e The 3 factors most strongly associated with pupils’ achievement at both years 5 and 9 
in England were headteachers reporting that their schools placed an emphasis on 
academic success (positive and significant association) and pupils reporting 
disorderly behaviour in school and that they experienced bullying behaviour in 
schools (negative and significant associations). 

e The average achievement of year 9 pupils was more likely to be negatively impacted 
by disorderly behaviour in classrooms compared with pupils in year 5. 

e Across all aspects of discipline, orderliness, and bullying, there was a negative and 
significant association with pupils’ average achievement: the less that pupils were 
adversely impacted, the higher their average achievement. This finding typically 
applied to both year groups and subjects in England, with the exception of discipline 
for year 9 pupils. 

e In mathematics and science in both years 5 and 9, there was a positive and 
significant association between an emphasis on academic success and average 
achievement in England. 

e In England there was a negative and significant association between resource 
shortages and average achievement for year 5 pupils in science, but not in 
mathematics, and no significant relationship between resource shortages and 
average achievement for year 9 pupils in either subject. The difference in the average 
score between England’s pupils not affected and those somewhat affected by 
resource shortages was smaller than the score difference associated with this factor 
across all participating countries as a whole, apart from in year 9 science. 

e There was a negative and significant association between the extent to which year 5 
pupils were taught in schools with discipline problems and average achievement, but 
there was no significant difference in achievement for year 9 pupils by the extent to 
which they were taught in schools with discipline problems. In both years 5 and 9, 
there were no pupils taught in schools where teachers reported moderate to severe 
discipline problems. 

e The difference in average score between England’s year 5 and year 9 pupils in 
schools with hardly any discipline problems, and those with minor problems, was 
greater than all participating countries as a whole. 

e The majority of pupils in England were taught in schools where headteachers 
reported few problems with school discipline and which teachers reported to be safe 
and orderly. However, in year 9, 8% of pupils were taught in schools that were 
considered less than safe and orderly. 

e In 2019, the majority of year 5 and year 9 pupils in England reported that they never 
or almost never experienced bullying behaviours. There was a negative and 
significant association between the extent to which pupils experienced bullying 
behaviours and their average achievement in England. 

e There was a negative and significant association between the extent to which pupils 
reported disorderly behaviour and their average achievement in England. 

e The difference in the average score between England’s year 5 and year 9 pupils who 
reported disorderly behaviour in few or no lessons and those who reported it in most 
lessons was greater than in participating countries as a whole. More year 5 pupils in 
England reported disorderly behaviour in some lessons than the international average 
and more year 9 pupils reported disorderly behaviour in most lessons than the 
international average. 
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8.2 To what extent was academic success emphasised in the 
schools where pupils were taught? 


Headteachers responded to the following statements using a 5-point rating scale from 
‘Very high’ to ‘Very low’ — the same response options were used across both subjects. 


1) Teachers’ understanding of the school's curricular goals 

2) Teachers’ degree of success in implementing the school's curriculum 
3) Teachers’ expectations for student achievement 

4) Teachers’ ability to inspire students 

5) Parental involvement in school activities 

6) Parental commitment to ensure that students are ready to learn 
7) Parental expectations for student achievement 

8) Parental support for student achievement 

9) Students’ desire to do well in school 

10) Students’ ability to reach school's academic goals 

11) Students’ respect for classmates who excel academically 


Based on headteachers’ responses, scores were calculated and pupils assigned to 1 of 3 
categories. These related to the extent of the emphasis on academic success in the 
schools in which they were taught: 


e very high emphasis 
e high emphasis 
e medium emphasis’? 


This chapter focuses on mathematics, making reference to science only where there are 
notable differences between the 2 subjects. A full account of findings is reported in the 
TIMSS 2019 International Report. 


8.2.1 To what extent was academic success emphasised in the schools 
where year 5 pupils were taught? 


As shown in Figure 85 below, 80% of year 5 pupils in England were taught in schools 
that placed a very high (12%) or high (68%) emphasis on academic success. This total 
percentage was above the international average (62%). There was a significant positive 
association between a greater emphasis on academic success and higher average 
achievement. Pupils taught in schools that placed a very high emphasis on academic 
success had significantly higher average achievement than those taught in schools that 
placed a high or medium emphasis on academic success, while pupils taught in schools 
that placed a high emphasis on academic success had significantly higher average 
achievement than those taught in schools that placed a medium emphasis on academic 
success. Similar findings were reported for year 5 science. 


The difference between the average mathematics score of those taught in schools that 
placed a very high emphasis on academic success (605) and those taught in schools that 


79 For full methodological explanations see the TIMSS 2019 International Report. 
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placed a medium emphasis on academic success (534) was 71 scale points, above the 
international average (29). 


Figure 85: Percentages of year 5 pupils in categories of schools by emphasis on 
academic success (headteachers’ reports) and their average achievement in 
mathematics (England and international average) 
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Source: TIMSS 2019. 


More year 5 pupils in England were taught in schools with a very high emphasis on 
academic success than was the case in the highest-performing comparator countries, 
except in the Republic of Korea. The same was the case in comparison to pupils in the 
other English-speaking countries, apart from those in Ireland and Northern Ireland. More 
year 5 pupils in England were taught in schools with a very high emphasis on academic 
success than in any European comparator country. 


8.2.2 To what extent was academic success emphasised in the schools 
where year 9 pupils were taught? 


As shown in Figure 86 below, 79% of year 9 pupils in England were taught in schools 
that placed a very high (18%) or high (61%) emphasis on academic success. This total 
percentage was above the international average (57%). As in year 5, there was a 
significant positive association between a greater emphasis on academic success and 
higher average achievement. Pupils taught in schools that placed a very high emphasis 
on academic success had significantly higher average achievement than those taught in 
schools that placed a high or medium emphasis on academic success, however, average 
achievement for pupils taught in schools that placed a high or medium emphasis on 
academic success were not significantly different. Similar findings were reported for year 
9 science. 


The difference between the average mathematics score of those taught in schools that 
placed a very high emphasis on academic success (602) and those taught in schools that 
placed a medium emphasis on academic success (500) was 102 scale points, above the 
international average (69). 
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Figure 86: Percentages of year 9 pupils in categories of schools by emphasis on 
academic success (headteachers’ reports) and their average achievement in 
mathematics (England and international average) 
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Source: TIMSS 2019. 


More year 9 pupils in England were taught in schools where headteachers reported a 
very high emphasis on academic success compared to their peers in all of the highest- 
performing comparator countries, except the Republic of Korea, where the percentage 
was lower, and Singapore, were the percentage was the same. Zero per cent of pupils in 
Russia were taught in schools with a very high emphasis on academic success. More 
year 9 pupils in England were taught in schools with a very high emphasis on academic 
success than in any of the 4 other English-speaking countries and any European 
comparator country. 
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8.3 To what extent was schools’ reporting of resource 
shortage associated with pupils’ average achievement? 


Headteachers responded to the following statements using a 4-point rating scale from 
‘Not a lot’ to ‘A lot’. These were largely consistent across both subjects with only 
statement 5 in part B being different. 


A) General school resources: 


1) Instructional materials (e.g. textbooks) 

2) Supplies (e.g. papers, pencils, materials) 

3) School buildings and grounds 

) Heating/cooling and lighting systems 

5) Instructional space (e.g. classrooms) 

6) Technologically competent staff 

7) Audio-visual resources for delivery of instruction (e.g. interactive white boards, 
digital projectors) 

8) Computer technology for teaching and learning (e.g. computers or tablets for 
student use) 


B) Resources for mathematics/science instruction: 


1) Teachers with a specialisation in mathematics/science 
2) Computer software/applications for mathematics/science instruction 
) Library resources relevant to mathematics/science instruction 
4) Calculators for mathematics/science instruction 
5) Concrete objects or materials to help students understand quantities or 
procedures/science equipment and materials for experiments 


Based on headteachers’ responses, scores were calculated and pupils assigned into 1 of 
3 categories. These related to the extent to which the schools in which they were taught 
were affected by resource shortages: 


e not affected 
e somewhat affected 
e affected a lot® 


8.3.1 To what extent were year 5 pupils taught in schools affected by 
mathematics and science resource shortages? 


Figures 87 and 88 below show that, in both mathematics and science in 2019, 0% of year 
5 pupils were taught in schools that were affected a lot by resource shortages, below the 
international averages (6% in mathematics and 7% in science). The percentage of pupils 
in England not affected by shortages in mathematics was similar to science (23% 
compared with 21%). A smaller percentage of year 5 pupils in England were taught in 
schools not affected by resource shortages than the international average (3 percentage 
points below this average for both subjects). 


80 For full methodological explanations see the TIMSS 2019 International Report. 
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There was no significant difference between the average mathematics achievement for 
pupils who were taught in schools that were somewhat affected by resource shortages 
and those who were not affected by resource shortages, but the difference in average 
science achievement was significant. The difference between the average science score 
of those taught in schools that were not affected by resource shortages (557) and those 
taught in schools that were somewhat affected by resource shortages (532) was 25 scale 
points, above the international average (20). 


Figure 87: The percentage of year 5 pupils taught in schools affected by 
mathematics resource shortages (headteachers' reports) and their average 
achievement (England and international average) 
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Source: TIMSS 2019. 


Note 1: 0% of year 5 pupils in England were taught in schools affected by mathematics 
resource shortages. 
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Figure 88: The percentage of year 5 pupils taught in schools affected by science 
resource shortages (Headteachers' reports) and their average achievement 
(England and international average) 
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Source: TIMSS 2019. 


Note 1: 0% of year 5 pupils in England were taught in schools affected by science 
resource shortages 


More year 5 pupils in England were taught in schools that were somewhat affected by 
resource shortages compared to their peers in all of the highest-performing comparator 
countries, except Hong Kong. However, fewer pupils in England were in schools affected 
a lot by resource shortages compared to their peers in these countries. Fewer year 5 
pupils in England were taught in schools that were not affected by resource shortages 
than pupils in all of the other English-speaking countries, with the exception of Ireland 
and Northern Ireland. Similarly, this was the case compared with 7 out of the 10 
European comparator countries. The exceptions were in France, Germany and Italy, 
while in science this was the case for the same 3 countries and also for the Netherlands. 
However, in comparison to both the English-speaking and European comparator groups, 
only in Finland, Germany, Norway and the Netherlands were zero percentages recorded 
for pupils affected a lot by resources shortages, as in England. 


8.3.2 To what extent were year 9 pupils taught in schools affected by 
mathematics and science resource shortages? 


As shown in Figures 89 and 90 below, in both mathematics and science, approximately 
1% of year 9 pupils were taught in schools that were affected a lot by resource shortages 
in 2019, below the international averages. The percentage of pupils taught in schools that 
were not affected by resource shortages in mathematics was smaller than for science 
(34% compared to 37% respectively). These percentages were above the international 
averages by 4 and 7 percentage points respectively. 
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For year 9 pupils there were no significant differences between the average mathematics 
and science achievement for pupils who were taught in schools that were somewhat 
affected by resource shortages and those who were not affected by resource shortages. 
The difference between the average mathematics and science scores of those taught in 
schools that were not affected by resource shortages (530 and 533) and those taught in 
schools that were somewhat affected by resource shortages (515 and 516) was 15 and 
17 scale points respectively, below the international average (26). 


Figure 89: The percentage of year 9 pupils taught in schools affected by 
mathematics resource shortages (headteachers' reports) and their average 
achievement (England and international average) 
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Source: TIMSS 2019. 
Note 1. Approximately 1% of pupils were categorised as ‘Affected a lot’, an insufficient 
quantity from which to calculate a reliable average score. 
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Figure 90: The percentage of year 9 pupils taught in schools affected by science 
resource shortages (headteachers' reports) and their average achievement 
(England and international average) 
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Source: TIMSS 2019. 
Note 1. Approximately 1% of pupils were categorised as ‘Affected a lot’: an insufficient 
quantity from which to calculate a reliable average score. 


More year 9 pupils in England were taught in schools that were not affected by 
mathematics resource shortages compared to their peers in Chinese Taipei and Russia, 
while the reverse was the case compared to pupils from the remaining 4 highest- 
performing countries. In science, in comparison to the highest-performing countries, more 
pupils in England were taught in schools that were not affected by resource shortages 
compared to their peers in Chinese Taipei, Japan and Russia. 


In comparison to the other English-speaking countries, more year 9 pupils in England 
were taught in schools that were not affected by resource shortages in both subjects 
compared to their peers in Ireland only, with the reverse being the case in the remaining 
3 countries. This was also the case in comparison with pupils in Italy and Lithuania only, 
where fewer pupils were taught in schools that were affected by resource shortages. The 
reverse was the case for the remaining 4 European comparator countries, which reported 
higher percentages of pupils who were in schools not affected by resource shortages 
compared with pupils in England. 
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8.4 How did pupils and staff rate their school climates in 
terms of discipline, safety and orderliness and bullying? 


8.4.1 To what extent were pupils taught in schools with discipline 
problems? 


Headteachers responded to the following statements using a 4-point rating scale from 
‘Not a problem’ to ‘Severe problem’. Statement 11 (*) was not included in the year 5 set 
of statements. 


1) Arriving late at school 

2) Absenteeism (i.e. unjustified absences) 

3) Classroom disturbance 

4) Cheating 

5) Profanity 

6) Vandalism 

7) Theft 

8) Intimidation or verbal abuse among students (including texting, emailing, etc.) 
9) Physical injury to other students 

10) Intimidation or verbal abuse of teachers or staff (including texting, emailing, etc.) 
11) Physical injury to teachers or staff* 


Based on headteachers’ responses, scores were calculated and pupils were assigned to 
1 of 3 categories. These related to the extent to which the schools in which they were 
taught reported discipline problems: 


e hardly any problems 
e minor problems 
e moderate to severe problems®' 


This section focuses on mathematics, making reference to science only where there are 
notable differences between the 2 subjects. Full findings can be found in the TIMSS 2019 
International Report. 


As shown in Figure 91 below, in 2019, 0% of year 5 pupils were taught in schools where 
there were moderate to severe discipline problems, below the international average (8%). 
More year 5 pupils in England were taught in schools where there were hardly any 
discipline problems in both subjects than the international average (67% compared to 
60%). 


For year 5 pupils in England, there was a significant negative association between being 
taught in schools with more discipline problems and higher average achievement. Pupils 
taught in schools where there were hardly any problems had significantly higher average 
achievement than those taught in schools with minor problems. The difference between 
the average scores in mathematics and science of those taught in schools where there 
were hardly any problems (566 and 545) and those taught in schools with minor 
problems (540 and 521) was 26 and 24 scale points, above the international averages 
(14 and 15). Only 4 countries in the whole study recorded 0% of pupils being taught in 


81 For full methodological explanations see the TIMSS 2019 International Report. 
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schools with moderate to severe discipline problems: England, Lithuania, the 
Netherlands and Singapore. 


Figure 91: The percentage of year 5 pupils in schools reporting the extent of 
school discipline problems (headteachers' reports) and their average achievement 
in mathematics (England and international average) 
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Source: TIMSS 2019. 


Note 1: 0% of year 5 pupils in England were taught in schools where the headteacher 
reported moderate to severe discipline problems. 


In 2019, fewer year 5 pupils in England were taught in schools with hardly any discipline 
problems compared to their peers in any of the highest-performing comparator countries. 
More pupils in England were taught in schools with hardly any discipline problems 
compared to their peers in all of the English-speaking countries, except those in Ireland 
and Northern Ireland. More year 5 pupils in England were taught in schools with hardly 
any discipline problems compared to their peers in 7 of the European comparator 
countries, with those in the Netherlands, Lithuania and Spain being the exceptions. 


As in year 5, in 2019, 0% of year 9 pupils in England were taught in schools where there 
were moderate to severe discipline problems, below the international average of 11% 
(see Figure 92 below). However, compared to year 5, more pupils were taught in schools 
in which there were minor problems (57% compared to 43%). Fewer year 9 pupils in 
England were taught mathematics and science in schools where there were hardly any 
discipline problems than the international average (43% compared to 45%). 


For year 9 pupils in England, there was a significant negative association between being 
taught in schools with more discipline problems and higher average achievement. Pupils 
taught in schools where there were hardly any problems had significantly higher average 
achievement than those taught in schools with minor problems. The difference between 
the average score in mathematics and science of those taught in schools where there 
were hardly any problems (534 and 535) and those taught in schools with minor 
problems (511 and 512) was 23 scale points in both subjects, in line with the international 
averages (22 in both subjects). Only 4 countries in the whole study recorded 0% of year 
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9 pupils being taught in schools with moderate to severe discipline problems: Chinese 
Taipei, England, Hong Kong and Singapore. 


Figure 92: The percentage of year 9 pupils in schools reporting the extent of 
school discipline problems (headteachers' reports) and their average achievement 
in mathematics (England and international average) 
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Source: TIMSS 2019. 


In 2019, fewer year 9 pupils in England were taught in schools with hardly any discipline 
problems compared to their peers in any of the highest-performing comparator countries. 
More pupils in England were taught in schools with hardly any discipline problems 
compared to their peers in New Zealand and the United States from the 4 English- 
speaking countries. The reverse was the case in comparison to pupils in Australia and 
Ireland. More pupils in England were taught in schools with hardly any discipline 
problems compared to their peers in any of the European comparator countries except 
Lithuania. 
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8.4.2 To what extent were pupils taught in schools that were safe and 
orderly? 


Teachers responded to the following statements using a 4-point rating scale from ‘Agree 
a lot’ to ‘Disagree a lot’. 


1) This school is located in a safe neighbourhood 

2) | feel safe at this school 

3) This school’s security policies and practices are sufficient 

4) The students behave in an orderly manner 

5) The students are respectful of the teachers 

6) The students respect school property 

7) This school has clear rules about student conduct 

8) This school’s rules are enforced in a fair and consistent manner 


Based on teachers’ responses, scores were calculated that assigned pupils into 1 of 3 
categories. These related to the extent to which the schools in which they were taught 
were safe and orderly: 


e very safe and orderly 
e somewhat safe and orderly 
e less than safe and orderly®? 


In 2019, the majority of year 5 pupils were taught in schools that were very or somewhat 
safe and orderly. However, fewer year 5 pupils in both subjects in England were taught in 
schools that were very safe and orderly than the international average (55% compared to 
61%). Figure 93 below shows that 4% of pupils in England were taught in schools that 
were less than safe and orderly, the same as the international average. 


For year 5 pupils there was no significant difference between the average mathematics 
and science achievement by the extent to which pupils were taught in safe and orderly 
schools. The difference between the average mathematics and science score of those 
taught in schools that were very, somewhat and less than safe was not significantly 
different. The difference between the average mathematics and science scores of those 
taught in schools that were very safe (564 and 542) and those taught in schools that 
were less than safe (537 and 515) was 27 scale points in both subjects, above the 
international average (12 and 4). 


82 For full methodological explanations see the TIMSS 2019 International Report. 
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Figure 93: The percentage of year 5 pupils in schools for which teachers reported 
on the extent of their safety and orderliness and their average achievement in 
mathematics (England and international average) 
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Source: TIMSS 2019. 


In 2019, more year 5 pupils in England were taught in schools that were very safe and 
orderly compared to their peers in half of the highest-performing comparator countries 
(Chinese Taipei, Japan and the Republic of Korea). The reverse was the case in Hong 
Kong, Russia and Singapore. Compared to the English-speaking countries, more year 5 
pupils in England were taught in schools that were very safe and orderly compared to 
their peers in Canada and the United States, with the reverse being the case compared 
with pupils in the remaining 4 countries. In Ireland and Northern Ireland, 78% and 75% of 
pupils respectively were taught in schools that were very safe and orderly. More year 5 
pupils in England were taught in schools that were very safe and orderly compared to 
their peers in half of the European comparator countries (Finland, France, Germany, 
Poland and Sweden). The reverse was the case for the remaining half, with higher 
percentages of pupils taught in schools that were very safe and orderly. In Spain, 76% of 
pupils were taught in schools that were very safe and orderly. 


The majority of year 9 pupils were taught in schools that were very or somewhat safe and 
orderly although 8% in mathematics and 7% in science were taught in schools that were 
less than safe and orderly, similar to the international averages of 6% (see Figures 94 
and 95 below). While more year 9 pupils in England were taught in schools that were 
very safe and orderly in mathematics than the international average (57% compared to 
48%), fewer were in science (45% compared to 49%). 


For year 9 pupils there was no significant difference between the average mathematics 
and science achievement by the extent to which the schools in which pupils were taught 
were safe and orderly. The difference between the average mathematics and science 
score of those taught in schools that were very, somewhat and less than safe was not 
significantly different. The difference between the average mathematics and science 
scores of those taught in schools that were very safe (532 and 532) and those taught in 
schools that were less than safe (519 and 512) was 13 and 20 scale points, below the 
international averages (41 and 31). 
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Figure 94: The percentage of year 9 pupils in schools for which teachers reported 
on the extent of their safety and orderliness and their average achievement in 
mathematics (England and international average) 
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Source: TIMSS 2019. 


Figure 95: The percentage of year 9 pupils in schools for which teachers reported 
on the extent of their safety and orderliness and their average achievement in 
science (England and international average) 
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Source: TIMSS 2019. 


In 2019, more year 9 pupils in England were taught mathematics in schools that were 
very safe and orderly compared to their peers in all of the highest-performing comparator 
countries, except Singapore. However, in science, this was the case for only half of the 
countries (Chinese Taipei, Japan and the Republic of Korea). More year 9 pupils in 
England were taught mathematics in schools that were very safe and orderly compared 
to their peers in 2 of the other English-speaking countries (New Zealand and the United 
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States), while in science this was the case only in comparison to the United States. In 
mathematics, more year 9 pupils in England were taught in schools that were very safe 
and orderly compared to their peers in all of the European comparator group countries, 
while in science this was also the case except for pupils in Lithuania and Norway. 


8.4.3 To what extent did pupils experience bullying behaviours? 


Pupils responded to the following statements using a 4-point rating scale from ‘Never’ to 
‘At least once a week’. There were some variations in the number and phrasing of 
statements between year groups. 


Year 5: 


1) Made fun of me or called me names 

2) Left me out of their games or activities 

3) Spread lies about me 

4) Stole something from me 

5) Damaged something of mine on purpose 
6) Hit or hurt me (e.g. shoving, hitting, kicking) 
7) Made me do things | didn't want to do 

8) Sent me nasty or hurtful messages online 
9) Shared nasty or hurtful messages about me online 
10) Shared embarrassing photos of me online 
11) Threatened me 


Year 9: 


1) Said mean things about my physical appearance (e.g. my hair, my size) 2) Spread lies 
about me 

3) Shared my secrets with others 

4) Refused to talk to me 

5) Insulted a member of my family 

6) Stole something from me 

7) Made me do things | didn't want to do 

8) Sent me nasty or hurtful messages online 

9) Shared nasty or hurtful things about me online 

10) Shared embarrassing photos of me online 

11) Threatened me 

12) Physically hurt me 

13) Excluded me from their group (e.g. parties, messaging) 
14) Damaged something of mine on purpose 
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Based on their responses, scores were calculated and pupils assigned to 1 of 3 
categories. These related to the extent to which they experienced bullying behaviours: 


e never or almost never 
e about monthly 
e about weekly® 


In 2019, the majority of year 5 pupils (60%) never or almost never experienced bullying 
behaviours, just below the international average (63%). As Figure 96 shows, more pupils 
in England (34%) than the international average (29%) experienced such behaviours 
about monthly, while fewer (6%) experienced these about weekly, similar to the 
international average (8%). 


For year 5 pupils in England, there was a significant negative association between pupils 
experiencing bullying behaviours and higher average achievement. Pupils who never or 
almost never experienced bullying behaviours had significantly higher average 
achievement than those who experienced these about monthly or about weekly, while 
pupils who experienced bullying behaviours about monthly had significantly higher 
average achievement than those who experienced bullying behaviours about weekly. 
The difference between the average mathematics and science scores of those who never 
or almost never experienced bullying behaviours (566 and 545) and those who 
experienced bullying behaviours about weekly (513 and 505) was 53 and 40 scale points, 
below the international averages (61 and 66). 


Figure 96: The percentage of year 5 pupils who experienced bullying behaviours 
and their average achievement in mathematics (England and international average) 
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Source: TIMSS 2019. 


Fewer year 5 pupils in England compared to their peers in the majority of the highest- 
performing comparator countries never or almost never experienced bullying behaviours. 
Pupils in Russia and Singapore were the exceptions. More pupils in England never or 


83 For full methodological explanations see the TIMSS 2019 International Report. 
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almost never experienced bullying behaviours compared to their peers in half of the 
English-speaking countries (Australia, Canada and New Zealand), while the reverse was 
the case for pupils in the remaining 3 countries. Fewer year 5 pupils in England never or 
almost never experienced bullying behaviours compared to their peers in all of the 
European comparator countries, with the exception of Italy. 


In 2019, the majority of year 9 pupils (69%) never or almost never experienced bullying 
behaviours, similar to the international average (71%). Figure 97 shows that more pupils 
in England (26%) experienced such behaviours about monthly than the international 
average (23%), while as many as the international average experienced these about 
weekly (6%). The percentage of year 9 pupils who never or almost never experienced 
bullying behaviours was higher in comparison to year 5 pupils. 


For year 9 pupils in England, there was also a significant negative association between 
pupils experiencing bullying behaviours and higher average achievement. Pupils who 
never or almost never experienced bullying behaviours had significantly higher average 
achievement than those who experienced these about monthly or about weekly, while 
pupils who experienced bullying behaviours about monthly had significantly higher 
average achievement than those who experienced bullying behaviours about weekly. 
The difference between the average mathematics and science score of those who never 
or almost never experienced bullying behaviours (526 and 527) and those who 
experienced bullying behaviours about weekly (459 and 461) was 67 and 66 scale points, 
in line with the international average for mathematics (68), but below the international 
average for science (78). 


Figure 97: The percentage of year 9 pupils who experienced bullying behaviours 
and the average achievement in mathematics (England and international average) 
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Source: TIMSS 2019. 


Fewer year 9 pupils in England never or almost never experienced bullying behaviours 
compared to their peers in any of the highest-performing comparator countries except 
Singapore. More pupils in England never or almost never experienced bullying 
behaviours compared to their peers in half of the 4 English-speaking countries (Australia 
and New Zealand). Ireland had a higher percentage of pupils than England in this 
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category while the United States had the same percentage as England. Fewer year 9 
pupils in England never or almost never experienced bullying behaviours compared to 
their peers in any of the European comparator countries. 


8.5 To what extent did pupils report disorderly behaviour in 
their mathematics lessons? 


Pupils were asked to rate the extent to which they experienced the conditions described 
in each statement on disorderly behaviour in their mathematics lessons®. Responses 
were made using a 4-point rating scale from ‘Never’ to ‘Every or almost every lesson’. 


1) Students don’t listen to what the teacher says 

2) There is disruptive noise 

3) It is too disorderly for students to work well 

4) My teacher has to wait a long time for students to quiet down 
5) Students interrupt the teacher 

6) My teacher has to keep telling us to follow the classroom rules 


Based on their responses, scores were calculated which assigned pupils into 1 of 3 
categories. These related to the extent to which they reported disorderly behaviour in: 


e feworno lessons 
e some lessons 
e most lessons® 


As shown in Figure 98 below, in 2019, the majority (74%) of year 5 pupils reported 
disorderly behaviour in some lessons, above the international average (68%). Fewer year 
5 pupils in England reported disorderly behaviour in few or no lessons than the 
international average (11% compared to 18%), while the same percentage as the 
international average (14%) reported this in most lessons. 


For year 5 pupils in England, there was a significant negative association between pupils 
reporting disorderly behaviour to be more frequent and higher average mathematics 
achievement. Pupils who reported disorderly behaviour in few or no lessons had 
significantly higher average achievement than those who reported disorderly behaviour in 
some lessons or most lessons, while pupils who reported disorderly behaviour in some 
lessons had significantly higher average achievement than those who reported disorderly 
behaviour in most lessons. The difference between the average mathematics score of 
those who reported disorderly behaviour in few or no lessons (587) and those who 
reported disorderly behaviour in most lessons (530) was 57 scale points, above the 
international average (33). 


84 Equivalent questions were not asked about science lessons. 
85 For full methodological explanations see the TIMSS 2019 International Report. 
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Figure 98: The percentage of year 5 pupils who reported disorderly behaviour 
during mathematics lessons and their average achievement (England and 
international average) 
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Source: TIMSS 2019. 
Note 1: Percentages do not sum to 100% due to rounding 


Fewer year 5 pupils in England reported disorderly behaviour in few or no lessons 
compared to their peers in the highest-performing comparator countries (although there 
were no data for Singapore). More pupils in England reported disorderly behaviour in few 
or no lessons compared to their peers in all of the English-speaking countries, except in 
Ireland and Northern Ireland. More pupils in England reported disorderly behaviour in few 
or no lessons compared to their peers in all of the European comparator countries except 
Finland, Lithuania and Poland. 


As with year 5, in 2019, the majority of year 9 pupils in England reported that disorderly 
behaviour happened in some lessons, similar to the international average (63% 
compared to 65%). Figure 99 shows that fewer year 9 pupils in England reported 
disorderly behaviour in few or no lessons, than the international average (18% compared 
to 21%), while more reported this in most lessons (19% compared to 13%). 


For year 9 pupils in England, there was also a significant negative association between 
pupils reporting disorderly behaviour to be more frequent and higher average 
mathematics achievement. Pupils who reported disorderly behaviour in few or no lessons 
had significantly higher average achievement than those who reported disorderly 
behaviour in some lessons or most lessons, while pupils who reported disorderly 
behaviour in some lessons had significantly higher average achievement than those who 
reported disorderly behaviour in most lessons. The difference between the average 
mathematics score of those who reported disorderly behaviour in few or no lessons (554) 
and those who reported disorderly behaviour in most lessons (481) was 73 scale points, 
above the international average (36). 
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Figure 99: The percentage of year 9 pupils who reported disorderly behaviour 
during mathematics lessons and their average achievement (England and 
international average) 
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Source: TIMSS 2019. 


Fewer year 9 pupils in England reported disorderly behaviour in few or no mathematics 
lessons compared to their peers in the highest-performing comparator countries 
(although there were no data for Singapore). More pupils in England reported disorderly 
behaviour in few or no lessons than in Australia and New Zealand, while the reverse was 
the case compared with pupils in Ireland and the United States. More pupils in England 
reported disorderly behaviour in few or no lessons compared to their European 
comparator peers in Italy, Norway and Sweden, with the reverse being the case for pupils 
in Finland, France and Lithuania. 


8.6 To what extent were the school-related factors associated 
with pupils’ achievement? 


Figures 100 and 101 below compare the school-related factors from this chapter and the 
extent to which they were associated with pupils’ average achievement in England. This 
was achieved through comparing the average score for pupils in the highest and lowest 
categories to calculate a range (shown by the bars below). However, for 2 factors this 
was different as it was reported that 0% of schools were affected a lot by resource 
shortages and that in 0% of schools were there moderate to severe discipline problems. 
In these cases, the difference in pupil achievement was calculated through comparing 
average scores for the highest and middle categories. For example, in the case of the 
resource shortages factor, the calculation was of the average score difference between 
pupils in schools where it was reported that they were not affected and those where it 
was reported that they were somewhat affected. 


It should be noted that while associations between different factors and average 
achievement can be made, this does not mean the differences are causal. 


Figure 100 shows that 3 factors were more strongly associated with year 5 pupils’ 
achievement in both subjects: schools’ emphasis on academic success, followed by 
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disorderly behaviour and pupils’ experience of bullying behaviours. Schools’ emphasis on 
academic success and pupils’ experience of bullying behaviour were also most 
associated with year 5 pupils’ average achievement in 2015 (questions on disorderly 
behaviour were not included in the 2015 report). However, the difference in performance 
by schools’ emphasis on academic success in mathematics for 2019 was 17 scale points 
greater than in 2015 (71 compared to 54). The difference for pupils’ experience of 
bullying behaviours and mathematics average achievement in 2019 was nearly double 
the 2015 difference (54 scale points compared to 31). It should be acknowledged that 
there were some small variations in the statements used across the 2 cycles. 


Figure 100: Differences in pupil and school characteristics and average 
achievement in mathematics and science (England, Year 5) 
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Source: TIMSS 2019. 


Note 1. No data on disorderly behaviour in science lessons. 
Note 2: Numbers may differ from the main figures by 1 scale point due to rounding. 


As with year 5, the 3 factors most associated with year 9 pupils’ average achievement 
were schools’ emphasis on academic success, pupils reporting disorderly behaviour and 
pupils experiencing bullying behaviours. The last of these factors appeared to show an 
even greater negative association with pupils’ scores than in year 5. The average score 
for pupils at the lower end of the range (those experiencing bullying behaviours most 
frequently) was lower than for all other categories. The negative association between 
disorderly behaviour and year 9 pupils’ average scores was also more evident than for 
year 5 pupils. 


As in 2015, the largest difference in performance was associated with schools’ emphasis 
on academic success, although this difference was even larger in 2019 for mathematics 
(102 compared to 89). The difference in performance associated with pupils’ experience 
of bullying behaviours in 2019 was more than twice the difference for 2015 in 
mathematics (66 scale points compared to 28). However, there were variations in the 
questionnaire statements used in each cycle. While this scale point difference was similar 
to the international average in mathematics (68), it was above the differences recorded 
for all comparator countries for which upper and lower category scores were available. 
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In 2015, the extent to which schools were safe and orderly was the factor second most 
associated with pupils’ average mathematics achievement (a difference of 66 scale 
points). However, in 2019, this difference was 14 scale points, less than one-quarter of 
the 2015 difference, with no variations in the statements used in the teacher 
questionnaires. This scale point difference was below the international average in 
mathematics (41) and below the differences recorded for all comparator countries for 
which upper and lower category scores were available. 


Figure 101: Differences in pupil and school characteristics and average 
achievement in mathematics and science (England, Year 9) 
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Source: TIMSS 2019. 


Note 1. No data on disorderly behaviour in science lessons 
Note 2: Numbers may differ from the main figures by 1 scale point due to rounding 
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Chapter 9. Teachers and teaching 


This chapter summarises findings from teacher and pupil questionnaires on aspects of 
teaching and learning. 


The response rates for the teachers’ questionnaires in England were lower than in the 
majority of countries in TIMSS 2019. As a result, data on the experience of teachers were 
available for fewer than 70% of pupils in England, the threshold the IEA sets for its 
international exhibits. The response rates were taken into account in the analysis for this 
section and findings were included only if they were based on data for more than 50% of 
pupils. For this reason, some questions only analysed year 9 responses, as set out 
below. 


Chapter sections below focus on: 


e The responses from teacher questionnaires on the extent to which year 5 and year 
9 pupils were taught by teachers with different: 
o levels of experience 
o subject specialisms 
o job satisfaction 


e The responses from year 9 teachers on the extent to which year 9 pupils: 
o were taught by teachers with different professional development needs 
© accessed computers in mathematics and science lessons 
© were given computer-based activities in lessons to support their learning 
o took tests using computers 


e The responses from pupil questionnaires on the extent to which year 9 pupils 
spent time on weekly homework. 


Where there were particularly interesting comparisons to be drawn between pupils in 
England and their peers in other comparator group countries, these are discussed. 


The chapter also describes, in several sections, whether or not these factors are 
associated with higher or lower performance in the TIMSS assessments, although it is 
important to note that an association (or correlation) between 2 variables (such as level 
of engagement and average achievement) is not the same as causation (i.e. that one 
thing causes the other). 


The comparator countries referred to in this chapter are listed in section 1.5°°. 


86 Canada, Ireland, Germany, the Netherlands, Poland and Spain did not participate in the year 9 
study. Finland, France, Lithuania, Russia and Sweden did not participate in the questions on mathematics 
homework (covered in section 9.6). 
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9.1 Main findings 


Teachers in England had less teaching experience on average than those in most of 
the comparator countries; this was evident in years 5 and 9, in both mathematics and 
science, and mirrored 2015 findings. 

Year 9 pupils in England who were taught mathematics by teachers with 20 or more 
years’ experience had significantly higher average scores than those taught by 
teachers with fewer than 5 years’ experience. However, length of teacher experience 
was not associated with higher mathematics or science scores for year 5 pupils or 
higher science scores for year 9 pupils. 

Around a quarter of year 5 pupils were taught by teachers with mathematics or 
science as either their main area of study or joint area of study (with primary 
education). 

In England, pupils’ average mathematics scores were not significantly different 
depending on their year 5 teachers’ specialisms. However, for science, year 5 pupils 
taught by teachers with science but not primary education as their main area of study 
had significantly higher average achievement than other pupils. Similarly, pupils 
taught by teachers with a main area of study in primary education but not science had 
significantly higher average scores than pupils whose teachers’ main areas of study 
were both primary education and science, or other subjects besides primary 
education or science. 

Eighty-nine per cent of year 9 pupils in England were taught mathematics by teachers 
who had studied mathematics as either their main area of study or joint main area of 
study (with mathematics education). The equivalent figure for science was 98%. 
There were no associations between specialisms and average scores. 

When asked about professional development, year 9 teachers in England responded 
similarly in both mathematics and science, highlighting the need for more support to 
improve pupils’ critical thinking or problem-solving skills and integration of technology 
into their teaching practice. 

In England, there was a positive and significant association between teacher job 
satisfaction and year 9 pupils’ average mathematics scores. The average score for 
England’s pupils taught by teachers who were very satisfied was significantly above 
the score for pupils taught by teachers who were less than satisfied with their job. At 
year 5 and for science at year 9 there were no significant differences in pupils’ 
average scores by teacher satisfaction. 

The majority of year 9 pupils in England spent between 16 and 30 minutes per subject 
each week on mathematics and science homework tasks. The highest average 
achievement in mathematics and science was for pupils who spent between 31 and 
60 minutes on homework. Average achievement for these pupils was significantly 
higher than that of pupils who spent up to 15 minutes or 90 minutes or more on their 
homework. 

In England, there was no association between average achievement and year 9 
pupils having access to computers during mathematics and science lessons. Across 
all participating countries as a whole, pupils who had access to a computer in lessons 
had higher average scores than those who did not. 

About one-fifth of pupils in England participated in monthly activities on computers in 
both mathematics and science, either whole-class and/or to support specific pupil 
attainment groups. This proportion was lower than that found in the majority of 
comparator countries. 
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9.2 How experienced were teachers in England and how did 
this compare to other TIMSS countries? 


Based on teacher responses, pupils were assigned to 1 of 4 IEA-defined categories 
reflecting their teachers’ experience in years: 


20 years or more 

at least 10 but less than 20 years 
at least 5 but less than 10 years 
less than 5 years 


As shown in Figure 102 below, in 2019, more than half of England’s year 5 pupils were 
taught mathematics by teachers with fewer than 10 years’ experience (25% by teachers 
with fewer than 5 years’ experience and 31% by teachers with between 5 and 10 years’ 
experience). This was similar overall to 2015, although in 2015 35% were taught by 
teachers with fewer than 5 years’ experience and 22% by teachers with between 5 and 
10 years’ experience. The combined 2019 percentage (56%) was nearly double the 
international average (29%). Similarly, the percentage of pupils taught by teachers with 
more than 20 years’ experience (13%) was below the international average (40%). The 
same findings applied to year 5 science in 2019. 


For year 5 pupils in England, there was no significant difference in average mathematics 
and science achievement for different levels of teacher experience. However, across all 
participating countries as a whole, average achievement in both subjects for pupils who 
were taught by teachers with 20 years or more experience was significantly higher than 
those taught by teachers with fewer than 5 years’ experience. 


Figure 102: The percentage of year 5 pupils taught by teachers with different years 
of experience and their average achievement in mathematics (England and 


international average) 
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As shown in Figure 103 below, in 2019, more year 5 pupils in England (56%) were taught 
mathematics by teachers with fewer than 10 years’ experience compared to their peers 
from all comparator group countries. Fewer year 5 pupils in England compared to their 
peers from all comparator countries were taught these subjects by teachers with more 
than 20 years’ experience. The same findings applied to year 5 science. 


Figure 103: Percentages of year 5 pupils taught mathematics by teachers’ years of 
experience (teachers' reports) (England and comparator countries) 
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As shown in Figure 104 below, in 2019, the highest percentage of year 9 pupils were 
taught mathematics by teachers with at least 10 but fewer than 20 years’ experience 
(40%). This was greater than in 2015 when this was the case for 25% of pupils in 
England. Just over a quarter (26%) were taught by those with 20 or more years’ 
experience in 2019, which was below the international average (35%), although above 
the 2015 percentage for England (17%). Nearly one-fifth (19%) were taught by teachers 
with fewer than 5 years’ experience, above the international average (14%) but below the 
2015 percentage for England’s pupils (29%). 


In England, pupils taught by teachers with 20 or more years’ experience scored, on 
average, 55 scale points above those taught by teachers with fewer than 5 years’ 
experience; no other differences between scores were significant (see Figure 104 below). 
Similarly, across all participating countries as a whole, the lowest average achievement 
in mathematics was for pupils who were taught by teachers with fewer than 5 years’ 
experience. 


Figure 104: Percentages of year 9 pupils taught by teachers with different years of 
experience and pupils’ average achievement in mathematics (England and 
international average) 
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As shown in Figure 105 below, in 2019, fewer year 9 pupils in England were taught 
mathematics by teachers with 20 or more years’ experience compared to their peers in 
the highest-performing countries, except in Singapore. This was also the case for pupils 
in the 4 other English-speaking countries and any of the European comparator countries 
apart from Norway. 


Figure 105: Percentages of year 9 pupils taught mathematics by teachers with 
different years of experience (teachers' reports) (England and comparator 
countries) 
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In 2019, as shown in Figure 106 below, year 9 pupils were mostly taught science by 
teachers with at least 5 but fewer than 10 years’ experience and at least 10 but fewer 
than 20 years’ experience (both 34%). Just under one-fifth (18%) were taught by those 
with 20 or more years’ experience, just over half the international average (34%). 
Thirteen per cent were taught by teachers with fewer than 5 years’ experience, below the 
international average (15%). This was less than half the 2015 percentage for England’s 
pupils (29%). 


In 2019, there were no significant differences between England’s pupils’ average scores 
in any of the categories represented (see Figure 106). 


Figure 106: Percentages of year 9 pupils taught by teachers with different years of 
experience and pupils’ average achievement in science (England and international 


Percentage of pupils 


100 


80 


60 


40 


20 


507 


488 


on 


Less than 5 years 


average) 
Average achievement 

600 

529 524 520 
500 

491 491 492 
400 
300 
200 
32 34 100 

18 
T —t * 0 
At least 5 but less than 10 At least 10 but less than 20 years or more 
years 20 years 
"England International Average 


Source: TIMSS 2019. 


183 


As shown in Figure 107 below, in 2019, fewer year 9 pupils in England were taught 
science by teachers with 20 or more years’ experience compared to their peers in the 
highest-performing countries, except in Singapore. This was also the case in comparison 
with pupils in the 4 English-speaking comparator countries and all of the European 
comparator countries. 


Figure 107: Percentages of year 9 pupils taught science by teachers with different 
years of experience (teachers' reports) (England and comparator countries) 
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9.3 To what extent were pupils taught by specialist 
mathematics and science teachers? 


This section identifies the percentage of pupils in England who were taught by teachers 
who studied mathematics and science at a post-secondary level®’ and compares these 
pupils’ overall average achievement. Based on their teachers’ responses, year 5 pupils 
were allocated into 5 categories according to their teachers’ main area(s) of study: 


1. Primary education and mathematics (or science) 

2. Primary education but not mathematics (or science) 
3. Mathematics (or science) but not primary education 
4. All other areas of study 

5. No formal education beyond upper secondary 


8” Post-18 Higher Education. 
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Year 9 pupils were also allocated into 5 categories, based on their teachers’ main area(s) 
of study: 


1. Mathematics and mathematics education (or science equivalent) 

2. Mathematics (or science) but not mathematics education (or science education) 
3. Mathematics (or science) education but not mathematics (or science) 

4. All other areas of study 

5. No formal education beyond upper secondary 


In the context of England, teachers’ main area of study can be interpreted as their main 
degree subject. Where a teacher has studied another subject in addition to their main 
area of study this is considered to be an additional subject specialism. 


Year 5 


Figure 108 below shows that just over half of year 5 pupils in England were taught by 
teachers with primary education, but not mathematics, as their main area of study (56%), 
above the international average (43%). Fewer pupils in England were taught by teachers 
with specialisms in both primary education and mathematics (20%) compared to the 
international average (32%). 


Across all countries as a whole, the highest average achievement was for pupils taught 
by teachers with primary education but not mathematics as a main area of study. In 
England, there were no significant differences between scores in any of the categories. 


Figure 108: Percentages of year 5 pupils taught by teachers with different main 
areas of study in mathematics (England and international average) 
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Source: TIMSS 2019. 


In some of the highest-performing countries, higher percentages of pupils, compared to 
those in other comparator group countries, were taught by teachers with primary 
education and mathematics as their main areas of study (Hong Kong — 53%; Russia — 
41%; Singapore — 66%). This was also the case in Norway (63%). 


There were no clear associations across comparator countries in relation to year 5 pupils’ 
average mathematics scores and categories of teachers’ area of study. For example, 
pupils in Chinese Taipei taught by teachers with mathematics but not primary education 
as a main area of study achieved higher average scores compared to their peers taught 
by teachers in the other categories. However, in Hong Kong, pupils’ average scores were 
highest for those taught by teachers with other main areas of study. This inconsistency is 
also found across the other English-speaking countries and the European comparator 
countries. 


As shown in Figure 109 below, in 2019, nearly 3 times as many year 5 pupils in England 
were taught by teachers with primary education but not science as their main area of 
study (56%) compared with those taught by teachers with higher education in both 
primary education and science (19%). The percentage of pupils taught by teachers in the 
former category (56%) was also above the international average (44%), while those 
taught by teachers in the latter category (19%) was below the international average 
(28%). 


Across all countries on average, the highest average achievement was for pupils taught 
by teachers with a main area of study in primary education but not science. This was 
different in England where the highest average achievement was for pupils taught by 
teachers with science but not primary education as their main area of study. In England, 
pupils taught by teachers in this category achieved significantly higher average scores 
(582) than pupils in the other categories. Similarly, pupils taught by teachers with a main 
area of study in primary education but not science had significantly higher average 
scores than pupils taught by teachers with both science and primary education as main 
areas of study or by teachers with other main areas of study (542 compared with 522 and 
524). 
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Figure 109: Percentages of year 5 pupils taught by teachers with different main 
areas of study in science (England and international average) 


19 
Education and Science ES oP 
NN ss 
Education but not Science 
44 
8 
Science but not Education cS 43 
ens 17 
Other 5 
| = England 
No formal education beyond upper 0 International Average 
secondary 5 
0% 20% 40% 60% 80% 100% 


Percentage of pupils 


Source: TIMSS 2019. 


As for year 5 mathematics, a relatively high percentage of pupils in Singapore were 
taught by teachers with specialisms in both primary education and science (57%), while 
this was also the case in Sweden (70%). 


There was no clear pattern of associations between higher average year 5 pupil scores in 
science and the categories across the highest-performing countries. For example, in both 
Japan and Hong Kong, pupils’ average scores were highest for those taught by teachers 
with other main areas of study, while in Singapore, those taught by teachers with a 
specialist education in science but no specialism in primary education had the highest 
average scores. Across the European comparator countries, there were similarly no clear 
patterns of association. However, in all of the other English-speaking countries except 
Northern Ireland, pupils taught by teachers with a main area of study in both primary 
education and science had the highest average scores. 


Year 9 


Figure 110 below shows that, in 2019, most year 9 pupils in England were taught by 
teachers with either a main area of study in mathematics and mathematics education 
(43%) or in mathematics but not mathematics education (45%). In both cases, these 
were above the international averages. There were no significant differences in pupils’ 
average scores between all of the categories in England. 
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Figure 110: Percentages of year 9 pupils taught by teachers with different main 
areas of study in mathematics (England and international average) 
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In Russia and Singapore, both in the highest-performing group of countries, there were 
relatively high percentages of pupils taught by teachers with mathematics and 
mathematics education as their main areas of study (68% and 60% respectively). Such 
relatively high percentages were not found across the remaining highest-performing 
countries or those from the other comparator groups. 


In 3 of the highest-performing countries, the highest average scores were for pupils 
taught by teachers with main areas of study in mathematics and mathematics education: 
these were Chinese Taipei, Hong Kong and Russia. This was also the case for pupils in 
Australia and the United States, among the English-speaking countries, but not in Ireland 
or New Zealand. There were no clear associations across the European comparator 
countries as a whole. 


Figure 111 below shows that in 2019, similar to the case for year 9 mathematics, most 
year 9 pupils in England were taught science by teachers with main areas of study either 
in science and science education (44%) or in science but not science education (54%). 
Both of these percentages were above the international averages. No significant 
differences in average score existed between any of the categories. There was no clear 
pattern of associations between teachers’ main areas of study and pupils’ average 
achievement in England or across all countries as a whole. 
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Figure 111: Percentages of year 9 pupils taught by teachers with different main 
areas of study in science (England and international average) 
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Note 1. Fewer than 1 per cent of pupils had a teacher with a main area of study reported 
to be science education but not science. There were insufficient pupils from which to 
calculate a reliable average score in either the ’science education but not science’ or 
‘other’ categories. 


In both Singapore from the highest-performing countries and Australia from the English- 
speaking countries, there were relatively high percentages of pupils taught by teachers 
with science and science education as their main areas of study (57% and 65% 
respectively). Such relatively high percentages (over 50%) were not found in other 
comparator countries. 


There were no clear associations across countries from the different comparator groups 


between pupils taught by teachers from the different categories and higher average 
scores. 
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9.4 What did year 9 teachers in England consider their future 
professional development needs to be? 


This question was asked of both year 5 and year 9 teachers but the response rates for 
year 5 teachers covered fewer than 50% of year 5 pupils and so have not been included. 
However, they are available in the TIMSS 2019 International Report. 


Teachers were asked to identify areas in which they needed future professional 
development (teachers could indicate needing professional development in more than 
one area): 


1) Mathematics/science content 

2) Mathematics/science pedagogy/teaching 

3) Mathematics/science curriculum 

4) Integrating technology into mathematics/science teaching 

5) Improving pupils’ critical thinking or inquiry skills 

6) Mathematics/science assessment 

7) Addressing individual pupils’ needs 

8) Integrating science with other subjects (year 5 science teachers only) 
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Mathematics 


As shown in Figure 112 below, in 2019, year 9 pupils were taught mathematics by 
teachers who considered their professional development needs to be primarily in 2 areas: 
improving pupils’ critical thinking or problem-solving skills (61%) and integrating 
technology into mathematics instruction (57%). Mathematics pedagogy/instruction was 
considered to be the next greatest professional development need (44%), while the area 
of least need was mathematics content (28%). The other 3 areas were considered to be 
of similar importance (between 36% and 38%). 


Figure 112: Percentages of year 9 pupils in schools taught by mathematics 
teachers with certain CPD needs (England and international average) 
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Note 1: Teachers could indicate need for CPD in more than one area. 
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Science 


A similar pattern of professional development needs was found in science as in 
mathematics. As shown in Figure 113 below, in 2019, year 9 pupils were taught science 
by teachers who considered their professional development needs to be primarily in 2 
areas: improving pupils’ critical thinking or problem-solving skills (56%) and integrating 
technology into science instruction (55%). Science pedagogy/instruction and science 
assessment (both 44%) were also considered to be important professional development 
needs. Science content (34%) and curriculum (35%) were the areas of least need. 


Figure 113: Percentages of year 9 pupils in schools taught by science teachers 
with certain CPD needs (England and international average) 
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9.5 How satisfied were teachers in England with their jobs? 


Teachers responded to the following statements using a 4-point rating scale from ‘Very 
often’ to ‘Never or almost never’. These were consistent across both year groups and 
subjects. 


1) | am content with my profession as a teacher 
2) | find my work full of meaning and purpose 
3) | am enthusiastic about my job 

4) My work inspires me 

5) | am proud of the work | do 


Based on teachers’ responses, scores were calculated that assigned pupils into 1 of 3 
categories. These related to the extent to which they were taught by teachers who were: 


e very satisfied with their jobs 
e somewhat satisfied with their jobs 
e less than satisfied with their jobs® 


Year 5 


This section focuses on mathematics for year 5, making reference to science only where 
there are notable differences between the 2 subjects. Full findings can be found in the 
TIMSS 2019 International Report. 


As shown in Figure 114 below, in 2019, 41% of year 5 pupils in England were taught 
mathematics by teachers who were very satisfied with their jobs, below the international 
average (61%). The majority of year 5 pupils in England (57%) were taught mathematics 
by teachers who were somewhat satisfied with their jobs in 2019. One per cent of pupils 
were taught by teachers who were less than satisfied with their jobs in 2019, compared 
with 12% in 2015. This percentage was below the international average (5%). 


Across all participating countries as a whole, average scores in mathematics and science 
were higher for year 5 pupils taught by teachers who were very satisfied with their jobs 
compared with those taught by teachers who were somewhat satisfied with their jobs. In 
England, the difference between average scores was not significant. 


88 For full methodological explanations see the TIMSS 2019 International Report. 
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Figure 114: Percentages of year 5 pupils taught by teachers with different levels of 
job satisfaction and their average achievement in mathematics (England and 
international average) 
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Source: TIMSS 2019. 
Note 1. Fewer than 1% of pupils had a teacher who was less than satisfied with their job, 
an insufficient quantity from which to calculate a reliable average score. 


Fewer year 5 pupils in England were taught by teachers who were very satisfied with 
their job compared to their peers in any of the highest-performing countries except the 
Republic of Korea (mathematics and science) and Japan (science only). Fewer pupils in 
England were taught by teachers who were very satisfied with their job compared to their 
peers in all of the other English-speaking countries. This was also the case in 
comparison with the majority of pupils in the European comparator countries, with the 
exceptions being those in Finland, France, Germany and Poland (in science, Finland, 
France and Poland only). 


Year 9 


As shown in Figure 115 below, in 2019, 37% of year 9 pupils in England were taught 
mathematics by teachers who were very satisfied with their job, below the international 
average (54%) but above the percentage of England’s pupils for the same measure in 
2015 (29%). Six per cent were taught by teachers who were less than satisfied with their 
job, which was in line with the international average in 2019 (7%) and below the 14% of 
England’s pupils in 2015. 


In England, there was a positive and significant association between higher levels of job 


satisfaction and pupils’ higher average scores, although this was inconsistent across all 
participating countries as a whole. The average score for England’s pupils taught by 
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teachers who were very satisfied with their job (533) was significantly above the score for 
pupils taught by teachers who were less than satisfied with their job (440). This difference 
of 93 scale points was greater than the difference across all countries as a whole, which 
was only 3 scale points. The difference in 2015 for the same measure in England was 18 
scale points (523 compared with 505), indicating that the performance gap for pupils 
taught by teachers who were very satisfied with their job compared with those who were 
less than satisfied with their job was more than 5 times larger in 2019. However, as 
stated above, the percentage of pupils taught by teachers who were less than satisfied 
with their job was smaller in 2019 compared with 2015. 


Figure 115: Percentages of year 9 pupils taught by mathematics teachers with 
different levels of job satisfaction and their average achievement in mathematics 
(England and international average) 
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Source: TIMSS 2019. 


Fewer year 9 pupils in England were taught mathematics by teachers who were very 
satisfied with their job compared to their peers in the highest-performing countries, 
except in Japan. Similarly, fewer pupils in England were taught by teachers who were 
very satisfied with their job than pupils in all 4 of the other English-speaking countries. 
This was also the case in comparison to pupils in half of the European comparator 
countries (Italy, Norway and Sweden), with the reverse being the case compared to 
pupils in Finland, France and Lithuania. 


As shown in Figure 116 below, in 2019, 39% of year 9 pupils in England were taught 
science by teachers who were very satisfied with their job, below the international 
average (53%) but above the percentage of England’s pupils for the same measure in 
2015 (27%). Fifteen per cent were taught by teachers who were less than satisfied with 
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their job, which was nearly double the international average (8%) but again, fewer than 
for England’s pupils in 2015 (21%). 


Across all countries as a whole, there was no clear relationship between job satisfaction 
and pupils’ average scores. Pupils taught by teachers who were very satisfied with their 
jobs achieved higher average scores than pupils taught by teachers who were somewhat 
satisfied and less than satisfied with their jobs. However, pupils taught by teachers who 
were less than satisfied with their jobs achieved average scores higher than pupils taught 
by teachers who were somewhat satisfied with their jobs®®. In England there were no 
significant differences in the average scores by teacher satisfaction. 


Figure 116: Percentages of year 9 pupils taught by science teachers with different 
levels of job satisfaction and the average achievement in science (England and 
international average) 
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Source: TIMSS 2019. 


Fewer year 9 pupils in England were taught science by teachers who were very satisfied 
with their job compared to pupils in the highest-performing countries, except in Japan and 
Russia. More pupils in all of the other English-speaking countries, except in New 
Zealand, were taught science by very satisfied teachers compared to their peers in 
England. This was also the case for pupils in half of the European comparator countries 
(Italy, Norway and Sweden) with the reverse being the case for pupils in Finland, France 
and Lithuania. 


89 There were high standard errors in the less-than-satisfied category. 


196 


9.6 How much time do year 9 pupils spend on homework in 
mathematics and science each week? 


Year 9 pupils only were asked how many minutes they usually spent on homework given 
by their teacher for each subject. They were asked to select 1 of the following options: 


1. My teacher never gives me homework 
2. 1-15 minutes 

3. 16-30 minutes 

4. 31-60 minutes 

5. 61-90 minutes 

6. More than 90 minutes 


Mathematics 


The majority of year 9 pupils in England usually spent between 16 and 30 minutes per 
week on homework in mathematics (see Figure 117 below). The highest average 
achievement in mathematics was for pupils who spent 31-60 minutes per week on their 
homework (535). This was significantly higher than the average achievement in 
mathematics for those who spent no time on homework (477), those who spent 1-15 
minutes (501) on it, and those who spent more than 90 minutes (499), but was not 
significantly different from the average achievement for pupils who spent 16-30 minutes 
(530) or 61-90 minutes (521) on homework. 


Figure 117: Percentages of year 9 pupils spending different time on mathematics 
homework per week and their average achievement in mathematics (England and 
international average) 
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Source: TIMSS 2019. 


As shown in Figure 118 below, in 2019, approximately three-quarters of pupils in the 
majority of comparator countries, like those in England, spent up to 30 minutes on 
mathematics homework per week. Finland was the notable exception with 95% of its 
pupils spending up to 30 minutes. Pupils in half of the highest-performing countries spent 
more time than this on mathematics homework per week (Hong Kong, Russia and 
Singapore), as well as pupils in Italy and Norway. Relatively high percentages of pupils 
spent more than 1 hour per week in Italy and Singapore. In addition, relatively high 
percentages of pupils in Sweden and the Republic of Korea spent no time on 
mathematics homework per week. However, a relatively high percentage of pupils from 
the Republic of Korea received additional tuition in mathematics as shown in section 
10.6.2. 


Figure 118: Percentages of year 9 pupils by minutes of weekly mathematics 
homework (England and comparator countries) 
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As in mathematics, the majority of year 9 pupils in England spent between 16 and 30 
minutes per week on homework in science (see Figure 119 below). The highest average 
achievement in science was for pupils who spent 31-60 minutes per week on their 
homework (551). This was significantly higher than the average achievement in 
mathematics for those who spent no time on homework (454), those who spent 1-15 
minutes (492), 16-30 minutes (536), and those who spent more than 90 minutes (506), 
but was not significantly different from the average achievement for pupils who spent 61— 
90 minutes (542) on homework. 


Figure 119: Percentages of year 9 pupils spending different time on science 
homework per week and their average achievement in science (England and 
international average) 
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In 2019, as in England, approximately three-quarters of pupils in the majority of 
comparator countries spent up to 30 minutes on science homework per week (see Figure 
120 below). More pupils in Italy and Singapore spent more time than this on science 
homework, with 1 in 10 spending more than 1 hour per week. Relatively high 
percentages of pupils in 2 of the highest-performing countries (Japan and the Republic of 
Korea) spent no time on science homework per week. However relatively high 
percentages of pupils from both countries received additional tuition in science as shown 
in section 10.6.2. 
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Figure 120: Percentages of year 9 pupils by minutes of weekly science homework 
(England and comparator countries) 
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9.7 To what extent were computers used in the year 9 
classroom? 


9.7.1 Access to computers in year 9 mathematics and science lessons 


Questions about access to computers in lessons were asked of both year 5 and year 9 
teachers but the response rates for year 5 teachers covered fewer than 50% year 5 
pupils and so have not been included. However, they are available in the TIMSS 2019 
International Report. 
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Teachers of year 9 pupils in both subjects responded yes or no to a question asking 
whether computers were available for pupils to use in lessons. Those who responded 
‘yes’ were also asked to select 1 option from the following: 


e every pupil has a computer 
e the class has computers that the pupils can share 
e the school has computers that the class can sometimes share. 


Mathematics 
In 2019, as shown in Figure 121 below, nearly a quarter (24%) of England’s year 9 pupils 
had access to computers during their mathematics lessons, below the international 


average (37%). Fewer pupils in England had access to computers in lessons compared 
to their peers in any of the comparator group countries except in France and Italy. 


Figure 121: Percentages of year 9 pupils whose teachers reported access to 
computers for mathematics lessons (England and comparator countries) 
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As shown in Figure 122, the percentages of England’s year 9 pupils who had access to 
computers in mathematics lessons that the class could sometimes use (17%), that pupils 
could share in the class (4%) or who had individual access (13%) were all below 
international averages (28%, 11% and 17% respectively). Similarly, the availability of 
computers was lower in England than the international average. 


Across all participating countries on average, pupils who had access to computers had 
higher average mathematics scores than those who did not (495 compared with 487). In 
England, pupils’ average scores in 2019 for those who did and did not have this access 
were almost the same (521 and 522 respectively). Where pupils in England did have 
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access to computers, there were no significant differences in average scores between 
pupils who had individual access to a computer, pupils in classes that had computers that 
pupils could share or pupils in classes where the school had computers that the class 
could sometimes use. 


Figure 122: Percentages of year 9 pupils whose teachers reported access to 
computers for mathematics lessons (England and international average) 
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Some comparator countries had relatively high percentages of each pupil having access 
to a computer, for example, Australia (62%), Norway (73%) and Sweden (80%) 
compared with England (13%). The percentages of pupils in the highest-performing 
countries where each pupil had access to their own computer were below 10% with 2 
exceptions: Hong Kong (22%) and Singapore (19%). 


Science 
As shown in Figure 123, in 2019, approximately one-third of England’s year 9 pupils had 
access to computers during their science lessons, below the international average (48%). 


Fewer pupils in England had access to computers during science lessons compared to 
their peers in all of the comparator group countries. 
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Figure 123: Percentages of year 9 pupils whose teachers reported access to 
computers for science lessons (England and comparator countries) 
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As shown in Figure 124, the percentages of England’s year 9 pupils who had access to 
computers in science lessons that the class could sometimes use (29%), that pupils 
could share in the class (7%) or who had individual access (14%) were all below 
international averages (39%, 17% and 19% respectively). Similarly, the availability of 
computers was lower in England than the international average. 


In England, average scores for pupils who did and did not have access to computers 
during science lessons were similar (524 and 522). Across all participating countries as a 
whole, pupils who had this access had higher average scores than those who did not 
(496 compared with 486). Where pupils in England did have access to computers, in line 
with the pattern for mathematics, there were no significant differences in average science 
scores between pupils who had individual access to a computer, pupils in classes that 
had computers that pupils could share and pupils in classes where the school had 
computers that the class could use sometimes. 


203 


Figure 124: Percentages of year 9 pupils whose teachers reported access to 
computers for science lessons (England and international average) 
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Source: TIMSS 2019. 
Note 1: Teachers could provide more than 1 answer. 


Some comparator countries had relatively high percentages of each pupil having access 
to a computer, for example, Australia (67%), Norway (77%) and Sweden (87%) 
compared with England (14%). Fewer pupils in England had access to their own 
computer compared to half of the highest-performing countries, for example 17% in 
Japan and 30% in Hong Kong. The exceptions were Chinese Taipei (5%), the Republic 
of Korea (7%) and Russia (8%). 


9.7.2 The use of computers to support year 9 learning 


Analysis in this section looks at the percentage of year 9 pupils whose teachers reported 
that they organised activities on computers at least monthly to support learning in both 
subjects. In addition, for England, there is analysis of the extent to which these activities 
were organised for different pupil groups: the whole class, high-performing pupils, low- 
performing pupils and pupils with special needs. 


In 2019, around one-fifth of year 9 pupils in England were taught by teachers who 
reported that they organised activities on computers to support their learning in 
mathematics at least monthly, either at a whole class level or for the specific groups 
stated above. Similar percentages were found for year 9 science. 


In 2019, fewer pupils in England were taught by teachers who reported that they 
organised whole-class mathematics computer activities at least monthly compared to all 
of the comparator countries, except in France, Japan and the Republic of Korea (see 
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Figure 125 below). England’s percentage (20%) was also below the international average 
(31%). In the highest-performing countries other than Japan and the Republic of Korea, 
the percentages of pupils who had such activities organised were no higher than 30% 
compared with England’s 20%. The percentages of pupils in 3 of the 4 other English- 
speaking countries whose teachers organised such activities were more than treble the 
percentage of pupils in England (67% or more). 


Figure 125: Percentages of year 9 pupils whose teachers reported that they 
organised activities for the whole class on computers at least monthly to support 
mathematics learning (England and comparator countries) 
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In 2019, fewer pupils in England were taught by teachers who reported that they 
organised whole-class science computer activities at least monthly compared to all of the 
comparator countries, except in France (see Figure 126 below). England’s percentage 
(25%) was also below the international average (43%). In the highest-performing 
countries, the percentages of pupils who had such activities organised were all within a 
14-percentage-point range (37% to 51%), while the percentages of pupils in 3 of the 4 
other English-speaking countries were 80% or more. 
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Figure 126: Percentages of year 9 pupils whose teachers reported that they 
organised activities for the whole class on computers at least monthly to support 
science learning (England and comparator countries) 
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Source: TIMSS 2019. 
Note: There may be small differences between figures due to rounding 


9.7.3 The use of computers in year 9 testing 


Analysis in this section identifies the percentage of year 9 pupils whose teachers 
reported that tests are taken on computers: 


e once a month or more 
e once or twice a year 
e never 


As shown in Figure 127 below, in 2019 the majority of pupils in England never took 
mathematics tests on computers (71% — above the international average of 61%), while 
11% took these once a month or more and 18% took these once or twice a year. Most of 
the highest-performing countries had either the same (11%) or a lower percentage of 
pupils who took tests on computers once a month or more, with Russia the exception 
(25%). Japan (97%) and the Republic of Korea (94%) had the highest percentages of 
pupils who never took mathematics tests on computers, while almost half of pupils in the 
United States did so once a month or more. 


Figure 127: Percentages of year 9 pupils whose teachers reported the frequency 
with which mathematics tests were taken on computers (England and comparator 
countries) 
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As shown in Figure 128 below, in 2019 the majority of year 9 pupils in England never 
took science tests on computers (82% — above the international average of 61%), while 
5% took these once a month or more and 13% took these once or twice a year. Half of 
the highest-performing countries (Japan, Singapore and the Republic of Korea) had 
similarly low percentages to England of pupils who took tests on computers once a 
month or more. Japan (92%) and the Republic of Korea (83%) had relatively high 
percentages of pupils who never took science tests on computers, while just over half of 
pupils in the United States did so once a month or more. 
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Figure 128: Percentages of year 9 pupils whose teachers reported the frequency 
with which science tests are taken on computers (England and comparator 


countries) 
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Chapter 10. Home environment 


This chapter summarises findings from pupil questionnaires on aspects of the home 
environment that support their learning. 


Chapter sections focus on: 


the extent to which year 5 and year 9 pupils: 


o had access to information technology at home 
o had access to their own study desk at home 
o were absent from school 


and the extent to which year 9 pupils: 
o had resources at home for learning 


o attended additional tuition outside of school, and for what purposes 
o used the internet for homework, and for what purposes 


Where there are particularly interesting comparisons to be drawn between pupils in 
England and their peers in other comparator group countries, these are discussed. 

The chapter also describes whether or not these factors were associated with higher or 
lower performance in the TIMSS assessments, although it is important to note that an 
association (or correlation) between 2 variables (such as level of engagement and 
average achievement) is not the same as causation (i.e. that one thing causes the other). 
The comparator countries referred to in this chapter are listed in section 1.5%. 


10.1 Main findings 


A majority of year 9 pupils (78%) had some home resources for learning. 17% had 
many resources and only 5% few resources. More pupils in England had many 
resources for home learning than pupils in the highest-performing countries 
(except for Japan and the Republic of Korea). 

Year 9 pupils with many resources for home learning had significantly higher 
mathematics and science average scores than those with some or few resources 
for home learning. The difference between the average scores of those with many 
and those with few resources was 122 scale points in mathematics and 138 in 
science — both above the international averages. 

Most year 5 and year 9 pupils in England had access to a computer or tablet at 
home (96% and 98% respectively). This was higher than in 2015 (80% and 89%, 
respectively). 

Sixty-three per cent of year 5 pupils and 96% of year 9 pupils in England had their 
own mobile phones. 

In 2019, 96% of year 9 pupils in England used a computer for their homework. 
Year 9 pupils in England who used the internet for schoolwork mostly used it to 
access assignments posted by their teachers (78%) and find information to aid 
understanding (75%). The percentages of pupils using the internet to access 
assignments and find information to aid understanding have both increased 
between 2015 and 2019 (from 71% and 66% respectively in 2015). 


°° Canada, Ireland, Germany, the Netherlands, Poland and Spain did not participate in the year 9 study. 
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e The majority of year 5 and year 9 pupils in England had access to a study desk at 
home, with a larger proportion of year 9 pupils (86%) having this than year 5 pupils 
(75%). The proportion of year 5 and year 9 pupils with access to a study desk was 
among the lowest across all of the comparator countries. 

e The uptake of additional tuition in mathematics and science by year 9 pupils in 
England was low compared to other countries. The 6 highest-performing countries 
had the largest percentages of pupils receiving tuition in both subjects. 

e Year 9 pupils in England who attended additional tuition in mathematics and 
science to excel performed significantly better than those who attended tuition to 
keep up. However, both groups performed significantly less well than pupils who 
did not attend any tuition, although caution should be taken in interpreting the 
relationship as causal, since pupils might attend additional tuition based on their 
relatively low prior academic performance. 

e Most year 5 pupils (68%) and year 9 pupils (59%) reported that they were never, 
or almost never, absent from school in 2019 — both percentages were above the 
international average (61% and 55%). 

e For year 5 and year 9 pupils in 2019, there was a significant positive association 
between lower absence rate and higher achievement in both mathematics and 
science. The difference in pupils’ average score between those who were never or 
almost never absent and those who were absent once a week was 93 scale points 
for year 5 mathematics and 83 scale points for year 5 science — both above the 
international averages (64 and 66 scale points). At year 9, the difference was 82 
scale points for mathematics — below the international average (90) — and 92 scale 
points for science, similar to the international average (91). 


10.2 To what extent did year 9 pupils have resources at home 
for learning and how did these relate to average achievement? 


In England, only year 9 pupils were asked about the resources they had at home for 
learning. Pupils reported on the availability of 3 home-based resources using a series of 
statements”: 


e the number of books at home 

e whether they had their own room and an internet connection in the home, just 1 of 
these, or neither 

e the highest level of either parent’s education. 


°1 Full details of the methodology can be found in the T/IMSS 2019 International Report. 
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Based on their responses, pupils were assigned to 1 of 3 categories: 


e pupils with many resources 
e pupils with some resources 
e pupils with few resources. 


For year 9 pupils in England, there was a significant positive association between having 
more resources at home and higher average mathematics achievement (see Figure 129 
below). Pupils with many resources had significantly higher average achievement than 
those with some or few resources, while pupils with some resources had significantly 
higher average achievement than those with few resources. The difference between the 
average score of those with many (575) and those with few resources (453) was 122 
scale points, above the international average (113). 


Figure 129: Percentages of year 9 pupils with different home resources for learning 
and their average achievement in mathematics (England and international average) 
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Source: TIMSS 2019. 


As in mathematics, for year 9 pupils in England there was a significant positive 
association between having more resources at home and higher average science 
achievement (see Figure 130 below). Pupils with many resources had significantly higher 
average achievement than those with some or few resources, while pupils with some 
resources had significantly higher average achievement than those with few resources. 
The difference between the average scores of those with many (582) and those with few 
resources (444) was 138 scale points, above the international average (118). 
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Figure 130: Percentages of year 9 pupils with different home resources for learning 
and their average achievement in science (England and international average) 
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Source: TIMSS 2019. 


As shown in Figure 131 below, more pupils in England had many resources for home 
learning compared to the international average and pupils in the highest-performing 
countries, with the exception of Japan and the Republic of Korea (in the latter, 40% of 
pupils had many resources compared to 17% in England). The majority of the highest- 
performing countries also reported relatively high percentages of pupils with few 
resources when compared to England, with the exceptions once again of Japan and the 
Republic of Korea. Fewer pupils in England had many resources compared to their peers 
in the other English-speaking countries, apart from New Zealand (where the percentage 
was the same). Fewer pupils in England also had many resources in comparison with 
pupils in the 3 Scandinavian countries (Finland, Norway and Sweden). However, more 
pupils in England had many resources compared to pupils in France, Italy and Lithuania, 
although the percentages of pupils with many resources in these countries were only just 
below the figure for England. 
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Figure 131: Percentages of year 9 pupils reporting the amount of resources they 
had at home for learning by country (England and comparator countries) 
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10.3 To what extent did pupils in England have access to 
information technology at home? 


Year 5 and 9 pupils were asked whether they had any of the following at home: 


e computer or tablet 
e internet connection 
e their own mobile phone. 


Most year 5 pupils had access to either a computer or a tablet at home in 2019, both in 
England and across comparator countries. In England, 96% of year 5 pupils and 98% of 
year 9 pupils reported that they had access to these (see Figures 132 and 133 below), 
above the international averages. Both percentages were higher compared to the 2015 
study (80% and 89% respectively). 
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The highest percentages of year 5 and year 9 pupils with access to a computer or tablet 
were in the Scandinavian countries (Finland, Norway and Sweden). In Russia, only 76% 


of year 5 pupils had such access, although in year 9 it was 96%. 


Figure 132: Percentages of year 5 pupils who had a computer or tablet at home 
(England and comparator countries) 


International Average 87 
Norway 99 
Sweden 98 
Finland 98 
Netherlands 98 
Canada 97 
96 
Northern Ireland 96 
Lithuania 96 
Poland 95 
France 95 
Singapore 95 
Hong Kong 95 
Spain 95 
Australia 95 
Germany 94 
lreland 94 
Republic of Korea 93 
Italy 93 
New Zealand 93 
United States 92 
Chinese Taipei 88 
Japan 88 
Russia 76 
0% 20% 40% 60% 80% 100% 


Percentage of pupils 
Source: TIMSS 2019. 


214 


Figure 133: Percentages of year 9 pupils who had a computer or tablet at home 
(England and comparator countries) 
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Source: TIMSS 2019. 


In 2019, 63% of England’s year 5 pupils and 96% of year 9 pupils had their own mobile 
phone (see Figures 134 and 135 below). This was just below the international average 
for year 5 pupils, but above the international average for year 9. As with the question 
about computer and tablet access at home, the Scandinavian countries again reported 
the highest percentages of pupils with their own mobile phones in both years 5 and year 


More year 5 pupils in England had their own mobile phone compared to their peers in 

France (36%), Italy (50%) and Spain (47%), with the remaining 7 European comparator 
countries recording higher percentages than in England. Fewer year 9 pupils in England 
had their own mobile phone compared to their peers in any of the European comparator 


countries, except France. 


There was variation in the percentage of year 5 pupils from the highest-performing 
countries who had their own mobile phone, ranging from Japan (47%) to Russia (91%). 
Although mobile phone ownership was higher for year 9 pupils in these countries (over 
90% for 5 of the 6 countries), Japan had a relatively low percentage of year 9 pupils with 


their own mobile phone (75%). 
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More year 5 pupils in England had their own mobile phone compared to their peers in all 
of the other English-speaking countries, while in year 9 this was also the case, except 
compared with pupils in Ireland. 


Figure 134: Percentages of year 5 pupils who had their own mobile phone (England 
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Figure 135: Percentages of year 9 pupils who had their own mobile phone (England 
and comparator countries) 
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Source: TIMSS 2019. 


10.4 What proportion of pupils in year 9 used the internet for 
schoolwork and for what purposes? 


Year 9 pupils were asked whether or not they used the internet for homework and, if they 
did, for what purposes. 


In England, 99% of year 9 pupils reported that they had an internet connection at home in 
2019, with very high figures also reported in all the comparator countries. 


Figure 136 shows that 96% of year 9 pupils in England used the internet to support 
homework, similar to the international average (95%). Most other comparator countries 
reported high use, with over 90% of pupils using the internet to support homework. Japan 
was the exception with only 72% of year 9 pupils reporting the use of the internet for this 


purpose. 
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Figure 136: Percentages of year 9 pupils who used the internet for schoolwork at 
home (England and comparator countries) 
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Lithuania 100 
Sweden 99 
Russia 98 
Singapore 98 
Italy 98 
France 97 
United States 96 
Australia 96 
Ireland 95 
New Zealand 95 
Finland 95 
Hong Kong 92 
Chinese Taipei 92 
Republic of Korea 91 
Japan 72 
0% 20% 40% 60% 80% 100% 


Percentage of pupils 


Source: TIMSS 2019. 
Note 1. In Lithuania, due to rounding, the percentage of pupils is shown as 100%. 


As shown in Figure 137 below, with respect to homework, year 9 pupils in England in 
2019 mostly used the internet to access assignments posted by their teachers (78%) and 
to find information to aid understanding (75%), both above the international averages. 
These were the also the most common uses of the internet for schoolwork reported by 
pupils in the 2015 study. However, the percentages of pupils using the internet to access 
assignments and find information to aid understanding had both increased between 2015 
and 2019 (from 71% and 66% respectively in 2015). As in 2015 (33%), the lowest 
percentage in 2019 was recorded for pupils communicating with their teacher (39%), 
below the international average (44%). Fewer pupils in England used the internet to 
collaborate with classmates on assignments or projects compared to the international 
average (54% compared to 77%). 
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Figure 137: Percentages of year 9 pupils using the internet for different types of 
schoolwork at home (England and international average) 
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Source: TIMSS 2019. 
Note 1: Pupils could identify more than 1 type of schoolwork at home. 


10.5 Did pupils in England have access to a study desk at 
home? 


Pupils were asked whether they had a study desk/table for their own use at home. 


In 2019, 75% of England’s year 5 pupils had a study desk/table for their own use 
compared to 86% of year 9 pupils (see Figures 138 and 139 below). While the year 5 
percentage was below the year 5 international average (80%), the year 9 percentage 
was above the equivalent average for year 9 (83%). Both proportions showed an 
increase compared to 2015 (71% for year 5 and 78% for year 9). 


Fewer year 5 and 9 pupils in England had a study desk/table for their own use compared 
to their peers in any of the comparator countries, apart from in New Zealand and the 
United States (with the addition of Italy in year 5). European countries tended to have a 
higher proportion of pupils who had a study desk/table for their own use and English- 
speaking countries a lower percentage. 
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Figure 138: Percentages of year 5 pupils who had a study desk/table for their own 
use at home (England and comparator countries) 
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Figure 139: Percentages of year 9 pupils who had a study desk/table for their own 
use at home (England and comparator countries) 
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Source: TIMSS 2019. 
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10.6 Did performance differ by year 9 pupils’ attendance of 
additional tuition outside of their school? 


Year 9 pupils only were asked to select 1 of 3 statements to determine whether they 
attended additional tuition outside of school and, if they did, for what purpose: 


e yes, to excel in class 
e yes, to keep up in class 
e no 


Pupils also reported the number of months they had attended either extra lessons or 
tutoring in mathematics and/or science outside of school in the previous 12 months. 


10.6.1 What proportion of pupils in England and comparator countries attended 
additional mathematics or science tuition outside of school? 


In 2019, 17% of England’s year 9 pupils attended additional mathematics tuition outside 
of school. In science, this was 10% of pupils. In both cases, the percentages were below 
the international averages. 


In 2019, 9% of pupils in England attended additional tuition in mathematics in order to 
excel in class, while 8% did so to keep up; these percentages were similar to 2015. 
Fewer pupils in England attended additional tuition in mathematics compared to their 
peers in any of the comparator countries except in France and Ireland in 2019 (see 
Figure 140 below). The 6 highest-performing countries had the highest percentages of 
pupils attending additional tutoring, ranging from Russia (35%) with more than double the 
percentage of pupils in England to the Republic of Korea (78%) with more than 4 times 
England’s percentage. 


The percentage of pupils attending extra lessons to excel in class was broadly similar 
across the majority of comparator countries (ranging from 6% in Ireland and 9% in 
England to 14% in Lithuania and Hong Kong). However, Chinese Taipei, Japan, the 
Republic of Korea, Russia and Singapore reported much higher percentages of pupils 
attending extra lessons or tuition to excel in class. The proportion of pupils attending 
extra lessons or tuition to keep up in class varied from 8% in England and France to 32% 
in the Republic of Korea and 34% in Hong Kong. 
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Figure 140: Percentages of year 9 pupils who attended extra lessons or tutoring in 
mathematics outside of school (England and comparator countries) 
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Source: TIMSS 2019. 


In 2019, 10% of England’s year 9 pupils attended additional science tuition outside of 
school, with equal percentages doing so to excel in class and keep up (see Figure 141 
below). These percentages were similar to 2015. Both in England and other comparator 
countries, fewer pupils attended additional tuition for science compared to mathematics. 


As seen in mathematics, the 6 highest-performing countries had the largest percentages 
of pupils attending additional tuition, ranging from Russia (19%) with nearly double the 
percentage of pupils in England to Chinese Taipei (39%) with nearly 4 times England’s 
percentage. Similar percentages of pupils in the English-speaking countries (9 or 10%) 
attended additional tuition in science compared with their peers in England, apart from in 
Ireland (4%). 
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Figure 141: Percentages of year 9 pupils who attended extra lessons or tutoring in 
science outside of school (England and comparator countries) 
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Source: TIMSS 2019. 


In 2019, fewer pupils in England attended additional mathematics tuition outside of 
school for more than 8 months, between 4 and 8 months, and for less than 4 months 
compared to the international averages and to their peers in the highest-performing 
countries (see Figure 142 below). The percentage of pupils attending tuition for more 
than 8 months in these countries ranged from 8% in Russia to 57% in the Republic of 
Korea (compared to 5% in England). More of these countries’ pupils attended tuition for 
more than 8 months compared with the 4-8 months or fewer than 4 months, except in 
Russia. In Russia, as in England, more pupils attended tuition for fewer than 4 months. 
However, more than twice as many pupils in Russia attended for this duration compared 
with pupils in England. 


Apart from in France and Ireland, more pupils in the English-speaking and European 


countries attended additional tuition for fewer than 4 months compared to pupils in 
England. 
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Figure 142: Percentages of year 9 pupils reporting the number of months they 
attended extra lessons or tutoring in mathematics outside of school in the 
previous 12 months (England and comparator countries) 
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Source: TIMSS 2019. 


In both England and comparator countries, fewer pupils attended additional tuition for any 
period of time in science compared to mathematics. In 2019, as in mathematics, fewer 
pupils in England attended additional science tuition outside of their school for more than 
8 months compared to the international average and their peers in the highest-performing 
countries (see Figure 143 below). Compared to 2% of England’s pupils in this category, 
the percentages in other comparator countries ranged from 6% in Hong Kong to 28% in 
Chinese Taipei (14 times the percentage for England). Similar percentages of pupils in 
England attended tuition for the different duration categories compared with their peers in 
most of the comparator countries. 
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Figure 143: Percentages of year 9 pupils reporting the number of months they 
attended extra lessons or tutoring in science outside of school in the previous 12 
months (England and comparator countries) 
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10.6.2 Was there an association between engagement in additional tuition outside 
of school and average achievement in mathematics and science? 


Among year 9 pupils in England in 2019, there was a significant negative association 
between attendance at additional tuition in mathematics and/or science and average 
achievement in the subject; pupils who did not attend additional tuition scored 
significantly higher on average (see Figure 144 below). This association was also found 
in 2015 for England’s pupils. This does not necessarily mean there was a relationship 
between non-attendance of additional tuition and higher average achievement more 
broadly. Pupils might have attended additional tuition based on their relatively low prior 
achievement compared to their peers, therefore reducing the overall average score for 
those who attended. 


Year 9 pupils attending additional tuition to excel had significantly higher average 
achievement in both subjects, compared to those attending to keep up. Compared to 
pupils who did not attend mathematics tuition, those who attended for less than 4 months 
had significantly lower mathematics achievement. However, there were no significant 
differences in achievement between pupils who did not attend additional tuition and those 
who attended tuition for 4-8 months or more than 8 months. Similarly, compared to 
pupils who did not attend science tuition, pupils who attended tuition for less than 4 
months or 4—8 months had significantly lower science achievement, but this was not the 
case for pupils who attended tuition for more than 8 months. 
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Figure 144: The average achievement score in mathematics/science by whether 
year 9 pupils attended extra lessons or tutoring in mathematics/science outside of 
school by reason for attending and duration of attendance (England) 
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10.7 How often were pupils absent from school? 


Year 5 and 9 pupils reported the extent to which they were absent from school in both 
subjects by selecting 1 of the following statements: 


never or almost never 
once every 2 months 
once a month 

once every 2 weeks 
once a week 


As shown in Figure 145 below, most year 5 pupils in England (68%) reported that they 
were never or almost never absent from school in 2019, above the international average 
(61%), but below their peers in all of the highest-performing countries. However, more 
pupils in England reported they were never or almost never absent from school than 
pupils in all of the English-speaking countries, and half the European countries (Finland, 
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Italy, Norway, Poland and Sweden), while more pupils reported this in the remaining 5 
European countries than in England. 


Figure 145: Percentages of year 5 pupils reporting levels of school absence 
(England and comparator countries) 
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As shown in Figure 146 below, most year 9 pupils in England (59%) reported that they 
were never or almost never absent from school in 2019, above the international average 
(55%), but below the averages in the 5 East Asian countries among the highest- 
performing countries. However, more pupils in England reported they were never or 
almost never absent from school compared to pupils in Russia, any of the 4 other 
English-speaking countries and any of the European comparator countries except 
France. 
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Figure 146: Percentages of year 9 pupils reporting levels of school absence 
(England and comparator countries) 
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10.7.1 Was there an association between absence and attainment? 


In 2019, there was a significant positive association between lower absence rates and 
higher achievement in both mathematics and science for year 5 pupils in England (see 
Figure 147 below). In England, the difference in pupils’ average score in mathematics 
between those who were never or almost never absent and those who were absent once 
a week was 93 scale points (565 compared to 472), above the international average (64). 
In science, this difference was 83 scale points (544 compared to 461), above the 
international average (66). 
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Figure 147: The percentage of year 5 pupils reporting school absence and their 
average achievement in mathematics and science (England) 
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In 2019, there was a positive association between lower absence rates and higher 
achievement for year 9 pupils in England in both mathematics and science (see Figure 
148 below). This was also the case across all participating countries. In England, the 
difference in pupils’ average score in mathematics between those who were never or 
almost never absent and those who were absent once a week was 82 scale points (529 
compared to 447), below the international average (90). In science, this difference was 
92 scale points (531 compared to 439), similar to the international average (91). 


229 


Figure 148: The percentage of year 9 pupils reporting school absence and their 
average achievement in mathematics and science (England and international 
average) 
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Chapter 11. Conclusion 


This report includes a range of evidence relating to the performance of pupils in England 
in mathematics and science, and the contextual conditions of that performance. The 
TIMSS International Report 2019 and encyclopedia explore these data and the 
international evidence in even greater detail. This conclusion concentrates on key issues 
and themes that have emerged from the TIMSS testing cycle in 2019 and highlights 
areas where further research might establish additional analytical insights. 


England has participated in TIMSS for 24 years — all of the cycles that the IEA has run — 
in both subjects and for both year groups”. 


In mathematics, from a relatively poor performance in both year groups in 1995, 
performance has considerably improved for pupils in England. Notably, pupil 
performance was significantly above the TIMSS centrepoint for both year groups, and for 
year 5 in 2019 it was the highest of any TIMSS cycle. The percentage of year 5 pupils 
reaching the low benchmark or above remained stable from 2015 to 2019, however, the 
percentage reaching the advanced benchmark and high benchmark or above increased 
significantly. The 2019 year 5 cohort of students in England has been taught according to 
the reformed national curriculum® since entering school in 2013. At year 9, performance 
in mathematics has been relatively stable since 2007. The 2019 year 9 cohort entered 
the education system in 2009, so was taught according to the new national curriculum for 
just over half the duration of its educational experience. 


Over the past 24 years England’s year 5 and year 9 pupils have consistently scored 
significantly above the TIMSS centrepoint in science®™. Performance for year 5 pupils 
has, for all cycles except the 2011 cycle, demonstrated continuous improvement over 
that time period. The performance of year 5 pupils remained broadly similar from 2015 to 
2019. However, for year 9 pupils the picture was different in 2019. For the first 20 years 
of TIMSS, England’s year 9 performance in science was stable, but in 2019 it dropped 
considerably, and this drop was to a score significantly lower than in any previous TIMSS 
cycle. In particular, the percentage of pupils performing below the low benchmark 
doubled from 2015. The reasons for this change are not obvious and require additional 
research. We propose the following as being worth further investigation: firstly, more 
schools are embarking on key stage 4 (GCSEs) in year 9 — a National Foundation for 
Educational Research (NFER) survey suggests that as many as half do so, and this 
change in practice could potentially lead to shifts in focus in the science content taught. 
Second, the year 6 key stage 2 tests in science have been replaced by a sampling 
regime°S, and research from the Wellcome Trust (2017) suggests this might have led to a 
reduced emphasis on science in primary schools’, with teachers perhaps focused more 
on ensuring pupils are prepared for the English and mathematics tests, which are still sat 


°2 The 1999 study was only run by the IEA for pupils in year 9. 

83 https://www.gov.uk/government/publications/national-curriculum-in-england-primary-curriculum and 
https://www.gov.uk/government/publications/national-curriculum-in-england-secondary-curriculum 

°4 The 1995 score was an average across the performance of two year groups (year 4 and year 5 pupils; 
year 8 and year 9 pupils). 

°5 https://Awww.nfer.ac.uk/education-inspection-framework-2019-nfer-response/ 

°6 https://www.gov.uk/government/publications/key-stage-2-science-sampling-tests 

87 Wellcome Trust (2017) ‘State of the nation’ report of UK primary science education. Available at: 
https://wellcome.org/reports/state-nation-report-uk-primary-science-education 
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by all pupils. Whether this potential shift in emphasis still affects pupils 3 years later is an 
area that might warrant further investigation. 


Although pupils from a group of mostly East Asian countries — Chinese Taipei, Hong 
Kong, Japan, Republic of Korea, Russia and Singapore — have consistently outperformed 
England’s pupils®®, we must not lose sight of the fact that pupils in England consistently 
perform significantly above the TIMSS centrepoint in both subjects and in both cohorts. 
England’s pupils also fared well when compared to their counterparts in other English- 
speaking countries — including Australia, Canada, Ireland, New Zealand, Northern Ireland 
and the United States — as well as when compared to pupils from the representative 
group of other European countries used in this report — Finland, France, Germany, Italy, 
Lithuania, the Netherlands, Norway, Poland, Spain and Sweden. 


Aside from the decline in year 9 science, performance issues were similar to those 
highlighted in 2015: between years 5 and 9 pupils’ scores did not increase and at times 
fell back, although we must be mindful that while the same cohort was tested in year 5 in 
2015 and year 9 in 2019, these were not necessarily the same pupils. Fewer pupils in 
England reached the advanced benchmark and high benchmark or above than those in 
the highest-performing countries, and wide achievement gaps remained between 
England’s most and least advantaged pupils. TIMSS 2019 data could be utilised in 
further study of the attainment gap between disadvantaged pupils and their peers, about 
which there is considerable, ongoing research. 


Performance across all domains in mathematics at year 5 was either stable or improved 
from 2015 — the improvement was significant in the data and number domains. The 
relative stability in overall scores in both year 9 mathematics and year 5 science was 
reflected in performance in the content and cognitive domains, which also remained 
stable, with a small number of exceptions (both increases and decreases). The most 
notable performance issue in 2019 was in year 9 science, where in all content domains 
(biology, chemistry, physics and Earth science) and cognitive domains pupils’ 
performance was significantly weaker than in 2015. In 2019 year 9 pupils were weakest 
in the reasoning domain, in contrast to 2015 when this was their strongest cognitive 
domain. 


In 2019 there were no significant differences between the performances of boys and girls 
across either subject and either year groups. In 2015, year 5 boys significantly 
outperformed girls in mathematics. 


Performance across ethnic groups did not differ significantly in mathematics in year 5 or 
year 9. In science in both year 5 and 9 pupils from other ethnic groups performed 
comparably to White British pupils, with the exception of Black pupils, who scored 
significantly below them. Further research into the performance of different ethnic groups 
in TIMSS could complement ongoing research into ethnic disparities in education, for 
example, the Office of National Statistics’ recent report on child poverty and education 
outcomes by ethnicity.°9 


88 With the exception of Hong Kong in science 

89 Antonopoulos, V., Dunkley, N., Radia, A., Santiago, L. and Williams, R. (2020) Child poverty and 
education outcomes by ethnicity. Office for National Statistics. Available at: 
https://www.ons.gov.uk/economy/nationalaccounts/uksectoraccounts/compendium/economicreview/februar 
y2020/childpovertyandeducationoutcomesbyethnicity 
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Performance by pupils for whom English was not their first language did not differ 
significantly in mathematics in both year 5 and year 9 or in science in year 5. In year 9 
science pupils whose first language was English significantly outperformed pupils for 
whom it was not. 


Pupils who were eligible for free school meals (FSM) at any time in the last 6 years 
performed significantly lower than their non-eligible peers across both year groups and 
both subjects. TIMSS’ proxy for socio-economic status, books at home, reveals a wide 
gap in performance for both year groups in both subjects. Again, these data could be 
interrogated further alongside the wider research on child poverty and its effects on 
educational attainment. 


Gender differences were clear in responses to questions asked about confidence in 
mathematics and science, as well as liking for the subjects. Overall, the more confident 
pupils were and the more they liked the subject, the better they performed in it. Girls were 
significantly less confident and liked the subject less in both year groups and for both 
subjects, but these negative attitudes did not generally manifest themselves in 
differences in achievement — girls’ outcomes were not significantly different from boys’. 


For both mathematics and science, teachers believed that their greatest professional 
development need was for support for sustained incorporation of technology into 
teaching and improved training to include problem solving and critical thinking in lessons. 
This could be something that mathematics and science organisations such as the 
National Centre for Excellence in the Teaching of Mathematics (NCETM) and the 
National Science, Technology, Engineering, and Mathematics (STEM) Learning Centre 
could review and consider for the programmes they coordinate. 


England’s schools compared favourably with international comparators as far as 
emphasis on academic performance, resources, discipline and safety were concerned. 
Many of England’s headteachers again reported that their schools focused on academic 
performance; this focus was reflected in stronger pupil performance. Most teachers 
believed that their schools maintained discipline well and none reported moderate to 
severe discipline problems. This finding was echoed by most pupils. When pupils 
reported that they were frequently bullied, or that their classes were frequently disrupted, 
their performance was weaker than when this was not the case. More year 5 pupils 
reported disorderly behaviour in some lessons than the international average and more 
year 9 pupils reported disorderly behaviour in most lessons than the international 
average; given the potential negative impact on performance, we propose that these 
issues warrant further exploration. 


Given England’s history of participating in the International Longitudinal Studies in 
Assessment (ILSAs), the research team have reflected on the TIMSS results alongside 
the recent outcomes from the PISA 2018. As mentioned in chapter 1, the two sets of 
tests emphasise different aspects of mathematics and science, with TIMSS concentrating 
more heavily on the intended curriculum and PISA on pupils’ ability to address real-life 
challenges. There is only 1—2 years’ (and 2 school years — years 9 and 11) difference 
between the ages of pupils tested, with TIMSS testing 13-14 year olds and PISA testing 
15 year olds. However, when comparing these data, it is very important to note that 
England’s pupils score above the OECD average in PISA assessments in mathematics 
and science, and that direct score comparisons are not possible. 
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Mathematics scores in PISA 2018 were significantly higher than in 2015, following many 
years of stable performance, and scores in science have remained relatively stable 
compared to past cycles, unlike in TIMSS 2019. Noting England’s pupils’ 2019 TIMSS 
outcomes in science at year 9, 12 OECD countries had significant decreases in PISA 
science scores between 2015 and 2018. Looking at the data for TIMSS 2019 science, 
year 9 pupils scored significantly below their overall average science score in the 
reasoning domain, and we propose some comparative research into question types 
might be useful in order to investigate potential reasons for such differences. 


In both TIMSS and PISA, it is the East Asian countries, including Chinese Taipei, Japan, 
the Republic of Korea and Singapore, that do best overall. Perhaps unsurprisingly, in 
both sets of assessments in England more advantaged pupils performed better than less 
advantaged ones and this trend has continued over time. 


Overall, the 2019 TIMSS results saw an improvement in year 5 pupils’ performance in 
mathematics, stability in year 9 mathematics and year 5 science, and a decline in year 9 
performance in science. These findings could indicate that more granular research needs 
to be conducted into the science outcomes, with the possibility of investigating the PISA 
science domains to ascertain further evidence. Additional research could complement 
already existing inquiries as to why it is, given the current emphasis placed on STEM 
subjects, that girls lack confidence in and enjoyment of mathematics and science. 
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Appendix A: Background 


TIMSS 2019 is the 7th cycle of the IEA’s'® series of comparative surveys of mathematics 
and science achievements. TIMSS has been administered every 4 years since 1995. The 
2019 survey provides an updated picture of participating countries’ educational 
performances relative to the previous study in 2015 and to some of the earlier cycles of 
TIMSS: 2007, 2003, 1999 (year 9 only) and 1995. 


TIMSS 2019 involved 64 participating countries and 8 benchmarking jurisdictions taking 
part at one or both of the target grades: 4'" and 8". In England, these grades correspond 
to years 5 and 9, with pupils aged 9-10 and 13-14 respectively. 


There were 58 countries and 6 benchmarking jurisdictions in the year 5 study and 39 
countries and 7 benchmarking jurisdictions in the year 9 study. 


Tables 20 and 21 contain a list of participating countries and benchmarking participants 
for years 5 and 9 for the TIMSS 2019 survey. The TIMSS participants are diverse, 
ranging from highly developed countries to developing ones, and include education 
systems representative of all major traditions (Comprehensive, selective, liberal, etc.). 
The countries vary in terms of the underlying characteristics of their education systems 
(e.g. age at which children start school, repetition, selection, length of each phase in 
number of years, etc.). More information about the educational systems in each country 
can be found in the TIMSS Encyclopedia™'. 


100 International Association for the Evaluation of Educational Achievement (IEA). http://www.iea.nl 
101 Mullis, |. V. S., Martin, M. O., Goh, S. & Cotter, K. (Eds.) (2019). TIMSS 2019 Encyclopedia: Education 
Policy and Curriculum in Mathematics and Science. Available at: httos://timssandpirls.bc.edu/timss2019/ 


235 


Table 21: Countries participating in TIMSS 2019 


Country Year 5 Year 9 Comparator country for 
England national report 

Albania Yes - - 
Armenia Yes - - 
Australia Yes Yes Yes 
Austria Yes - - 
Azerbaijan Yes - - 
Bahrain Yes Yes - 
Belgium (Flemish) Yes - - 
Bosnia and Yes - - 
Herzegovina 

Bulgaria Yes - - 
Canada Yes Yes 
Chile Yes Yes - 
Chinese Taipei Yes Yes Yes 
Croatia Yes - - 
Cyprus Yes Yes - 
Czech Republic Yes - - 
Denmark Yes - - 
Egypt - Yes - 
England Yes Yes - 
Finland Yes Yes Yes 
France Yes Yes Yes 
Georgia Yes Yes - 
Germany Yes - Yes 
Hong Kong Yes Yes Yes 
Hungary Yes Yes - 
Iran Yes Yes - 
lreland Yes Yes Yes 
Israel - Yes - 
Italy Yes Yes Yes 
Japan Yes Yes Yes 
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Country Year 5 Year 9 Comparator country for 
England national report 
Jordan - Yes - 
Kazakhstan Yes Yes - 
Kosovo Yes - - 
Kuwait Yes Yes - 
Latvia Yes - - 
Lebanon - Yes - 
Lithuania Yes Yes Yes 
Malaysia - Yes - 
Malta Yes - - 
Montenegro Yes - - 
Morocco Yes Yes - 
Netherlands Yes - Yes 
New Zealand Yes Yes Yes 
North Macedonia Yes - - 
Northern Ireland Yes - Yes 
Norway Yes Yes Yes 
Oman Yes Yes - 
Pakistan Yes - - 
Philippines Yes - - 
Poland Yes - Yes 
Portugal Yes Yes - 
Qatar Yes Yes - 
Republic of Korea Yes Yes Yes 
Romania - Yes - 
Russia Yes Yes Yes 
Saudi Arabia Yes Yes - 
Serbia Yes - - 
Singapore Yes Yes Yes 
Slovak Republic Yes - - 
South Africa Yes Yes - 
Spain Yes - Yes 
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Country Year 5 Year 9 Comparator country for 
England national report 


Sweden Yes Yes - 
Turkey Yes Yes - 

[United ArabEmirates [Yes [Yes |- © | 
United States Yes Yes Yes 


Table 22: Benchmarking jurisdictions participating in TIMSS 2019 


Jurisdictions Year 5 Year 9 
Ontario, Canada Yes Yes 
Quebec, Canada Yes Yes 
Moscow City, Russia Yes Yes 
Gauteng, South Africa | - Yes 
Western Cape, South | - Yes 
Africa 
Madrid, Spain Yes - 

Abu Dhabi, UAE Yes Yes 
Dubai, UAE Yes Yes 
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Appendix B: Methodology 


TIMSS sampling methodology 


The overall aim of the TIMSS methodology was to generate a sample of pupils 
representative of the year 5 and year 9 populations in England in 2019, who would 
participate in the study to yield accurate, unbiased and internationally comparable 
estimates of mathematics and science attainment and attitudes. 


A 2-stage sampling model was followed. In stage 1, schools were sampled from a list of 
schools in England, with each individual school having a higher probability of being 
chosen the larger its year-group cohort. Schools were also grouped by type and 
attainment to ensure national characteristics were proportionally represented in the 
sample. Stage 2 took the sample of schools and selected 1 or more classes at random, 
depending on the number of pupils in the school in the year group. All pupils in a selected 
class were expected to complete the study, other than those unable to participate. Pupil 
exclusions were kept to a minimum: only pupils with significant special educational needs 
that would limit them in following the instructions of the TIMSS tests, and pupils unable to 
read or speak English, were excluded. 


The technical specifications outlined by the IEA advise drawing a sample of 4,500 pupils 
from a minimum 150 schools per year group in each country. The aim is to assess at 
around 4,000 pupils via the onscreen eTIMSS system (though this figure is not a 
requirement), after allowing for some non-participation. A further 1,500 pupils were 
sampled to take part in a paper-based ‘bridging’ study used to link the eTIMSS 
assessment to the historic TIMSS assessment scale. These targets were designed to 
achieve a level of precision in the survey population at the 95% confidence interval of + 7 
score points for average achievement estimates and by confidence intervals of + 
3.5% for any percentage estimate at the country level. 


All schools in England with pupils aged 9 (year 5) or 13 (year 9) on 31 August 2018 were 
within the target population for sampling purposes. Schools were excluded at this stage 
on 2 criteria: 


e Schools with small year groups (fewer than 9 pupils in year 5, fewer than 25 pupils in 
year 9) 

e Special schools, pupil referral units and alternative provision schools where the 
majority of pupils did not follow the national curriculum 


TIMSS guidance stipulated that exclusions should be limited to a maximum of 5% of the 
total population across all stages of the survey. 


In TIMSS, each country had a main sample of schools and 2 matched replacement 
samples, which were included in the survey if the main sample schools declined to 
participate. Each school in the main sample was assigned a first and a second 
replacement school, which had the same key sampling characteristics. This ensured that 
if the main sample school declined to participate, it could be replaced with a similar 
school. This way, samples remained representative of the characteristics of the national 
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education systems they were drawn from even if some main sample schools did not 
participate. If a main sample school and its 2 replacement schools declined to participate 
then the participant country could not include any other school, to avoid skewing the 
sample. 


After the schools were sampled, the classes of pupils of the target age were then 
randomly selected with 95% of classes expected to participate. Within each class, 85% of 
pupils were expected to participate. The IEA’s sampling referee inspected samples and if 
they met the criteria they were accepted for TIMSS 2019. 


Table 23: Criteria for inclusion in TIMSS 


Criteria fully met Criteria partially met | Criteria not met 
A. Aminimum school participation | C. At least 85% of D. Fewer than 
rate of 85%, based on main schools, including 85% of schools 
sample schools. replacements, with at | including 
OR least 50% from the replacements 
B. Aminimum combined school, | Main sample 


classroom and student 
participation rate of 75%, based on 
main sample schools (although 
classroom and student 
participation rates include 
replacement schools). 


Participants that achieved either criterion A or B were deemed to have met their sampling 
requirements fully. Participants that achieved C were deemed to have achieved a sample 
that was suitably representative at national level, but data from the country would be 
annotated in the TIMSS International Report 2019, with a note to indicate that 
replacement schools were used, and might be required to conduct an analysis of 
potential bias in the TIMSS pupil sample compared to the national pupil cohort. 
Participants not meeting the criteria, in condition D, would have results reported 
separately in the /nternational Report and be expected to conduct a bias analysis. 


England’s TIMSS 2019 sample 


Schools in England were stratified by school funding type and prior attainment: 


e School type (local authority, academy, independent) 
e Attainment (KS2 or KS4 results split into quintiles, plus schools with no previous 
results) 


Exclusions were applied at school level and within schools in the same way for both the 
year 5 and year 9 samples. At school level, international schools, special schools and 
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very small schools were excluded, resulting in approximately 2% of the total eligible 
cohort being excluded (with the majority coming from special schools). Within schools, 
pupils with significant special needs and pupils unable to read or speak English were also 
excluded, resulting in a further <2% of the eligible cohort being excluded in each year 


group. 


The IEA stipulated that pupils from 301 English schools be selected to take part in TIMSS 
2019 (150 schools educating year 5 pupils and 151 educating year 9 pupils). The target 
school sample was provided by the IEA in August 2018 and schools were invited to 
participate in TIMSS 2019. Once the final sample of participating schools had been 
agreed, approved class sampling was conducted using IEA-supplied software. Two 
classes were sampled from schools with more than 90 year 5 pupils or 245 year 9 pupils 
on the sampling frame. 


For year 5 this resulted in 130 schools having a single sampled class and 20 having 2 
classes sampled. For year 9, these figures were 120 schools with a single sampled class 
and 31 schools with 2 sampled classes. 


In total, 139 year 5 (primary) schools were recruited — 129 from the main sample, 9 first 
replacement schools and 1 second replacement school. This total meant that an 86% 
sampled school participation rate was achieved, which exceeded participation criterion A, 
the 85% target set by the IEA, therefore ensuring that England was included within the 
TIMSS International Report 2019 without any caveats. With replacement schools 
included, a 93% participation rate was achieved. 


A total of 136 year 9 (secondary) schools were recruited — 125 from the main sample and 
11 first replacement schools. An 83% sampled school participation rate was achieved, 
just below the 85% target set by the IEA. With pupil and class participation included, a 
79% overall participation rate was achieved, exceeding the IEA’s criterion B, therefore 
ensuring that England was included within the T/MSS International Report 2019 without 
any caveats. When replacement schools were included, a 90% participation rate was 
achieved. Tables 23 and 24 below present the final number of schools recruited in 
England. 


Table 24: Year 5 TIMSS school sample in England 


Number | Number | Number of Total Number | Number | Number Total 
of main of first second number | of main of first of number 
sample | replace- replace- of sample | replace- | second of 
schools ment ment schools | schools ment replace- | schools 
recruited | schools schools | recruited | declined | schools ment declined 

recruited | recruited declined | schools 

declined 

129 9 1 139 21 12 11 44 
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Table 25: Year 9 TIMSS school sample in England 


Number | Number | Number Total Number | Number | Number Total 
of main of first of number | of main of first of number 
sample | replace- | second of sample | replace- | second of 
schools ment replace- | schools | schools ment replace- | schools 
recruited | schools ment recruited | declined | schools ment declined 
recruited | schools declined | schools 
recruited declined 
125 11 0 136 26 15 14 55 


In England, 3,396 year 5 pupils participated in the 2019 TIMSS assessments (a 96% 
pupil participation rate) and 3,365 year 9 pupils participated in the 2019 TIMSS 
assessments (a 95% pupil participation rate). In both year groups the participation rates 
exceeded the 85% pupil participation targets set by the IEA. The total number of 
assessed pupils was below the recommended level of 4,000 due non-participation at 
both school and pupil levels, however this did not affect the analysis of results. For more 
information on the schools and pupils that participated in TIMSS see Tables 2 and 3 in 
Section 1.3. 


It is important to note that although the study is designed to test a nationally 
representative sample of pupils, the class group(s) within the school that take part are 
randomly selected by the national centre, and might not necessarily be representative of 
all pupils in a sampled school (for example, in a secondary school where setting is 
applied in mathematics and either the top or bottom set has been selected to complete 
the assessment). One implication of this approach is that robust analysis cannot be 
undertaken by school type — for example, an academy that might have its top set for 
mathematics selected cannot be compared with a maintained school where the bottom 
set is selected. A second caveat to note is that the pupils who took TIMSS tests were 
selected from a stratified school sample rather than a stratified pupil sample. This means 
that it is not always possible to analyse TIMSS results for small sub-groups of pupils 
because it is likely that there are relatively few TIMSS pupils from some minority groups. 
The mathematics and science teachers for each class selected to take part in TIMSS 
2019, along with the headteachers from each of the participating schools, were also 
asked to fill in a questionnaire. 


In England, 97 teachers completed the year 5 questionnaire (a response rate of 55%) 
and 175 teachers completed the year 9 questionnaire (a response rate of 64%). 

The teacher participation rate in both years was below the 85% target set by the IEA and 
findings were annotated to indicate that the data available represented at least 50% but 
less than 70% of students. 


There was a 66% response rate to the year 5 headteacher questionnaire and a 71% 
response rate to the year 9 headteacher questionnaire. In both year groups the 
participation rate was below the target of 85% of pupils represented and the results were 
annotated to indicate that the data available represented at least 50% but less than 70% 
of pupils. 
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Survey administration 


Ahead of the sample selection process, a field trial took place in March 2018, in which 
school recruitment, new assessment questions and each background questionnaire 
(pupil, teacher and school) was trialled to identify whether the questions were likely to 
provide valuable information for the main study. 


Test materials were provided by the IEA and Pearson adapted the test items for use in 
England, involving amendments to spellings from American English and changes to 
subject-specific terminology to make it familiar to classroom usage in England. At the 
same time a small number of additional questions were added to teacher and 
headteacher questionnaires covering areas of particular interest in England. This process 
took place between December 2018 and January 2019, with similar exercises conducted 
in each participating country to ensure that tests provided fair and reliable assessments 
of pupils’ knowledge that could be compared internationally. Pearson also undertook a 
curriculum matching exercise to identify which of the TIMSS test items pupils in English 
schools would have been expected to have studied by the time of the TIMSS tests. 
Every participating school nominated a TIMSS school coordinator, who worked with a 
dedicated TIMSS test administrator from Pearson to ensure that tests were delivered to 
the IEA’s exact requirements. Any discrepancies in test delivery methods between 
countries could introduce bias into the study. 


Adherence to the study procedures to ensure consistency and fairness was monitored by 
quality monitors from the IEA and England, each visiting 10% of schools in each year 


group. 


For the main eTIMSS assessment, pupils were asked to complete mathematics and 
science test items onscreen using a tablet device supplied to schools (pupils taking the 
bridging study took tests on paper). The background questionnaires for all pupils were 
completed on paper whilst headteachers and teachers completed online questionnaires. 


With the agreement of the IEA, the main survey test period was staggered to best 
accommodate the two student populations. To avoid the summer examinations period, 
year 9 was assessed between 25 February and 5 April 2019. Year 5 started after the 
Easter break on 23 April and finished on 17 June 2019. 


The eTIMSS assessments were uploaded from the tablets to a central IEA server, with 
the questionnaires and administration documents couriered from schools for processing 
by Pearson. The data for England were submitted to the IEA for processing and checking 
before they were merged with the other participating countries’ data. The IEA also 
commissioned a TIMSS Encyclopedia article from each participating nation, which 
contained an overview of the structure of local education systems in participating 
countries. 


Data analysis 
The IEA analysed the international database of country results and the evidence from 


pupil, headteacher and teacher questionnaires. This analysis is available in the IEA’s 
TIMSS International Report 2019. The IEA released the international database 
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underpinning its report on 1 September 2020 and these data have been used to produce 
this report for England. 


The data for England have been linked to the to the national pupil database (NPD). The 
international and national reports were published on 8 December 2020. 


Coverage 


Throughout the report the year 5 mathematics and science achievement data are based 
on test results for the 3,396 year 5 pupils and 3,365 year 9 pupils who took part in 
eTIMSS 2019. 


The process of matching the TIMSS 2019 data and the records from the NPD resulted in 
a data set of 3,214 year 5 pupils and 3,193 year 9 pupils. Not all TIMSS 2019 pupils 
could be matched to a record on the NPD, in most cases because they attended 
independent schools and therefore did not have school census records. Analysis of the 
matched TIMSS 2019—-NPD data set in chapter 6 is therefore based on these samples of 
pupils. 


Sources of further information 


For more information on sample design and implementation, instrument development, 
translation, quality assurance, and creation of the international database visit: 
https://timssandpirls.bc.edu/timss2019/methods/index.html 


For documentation on methods and procedures in TIMSS 2019 refer to: 
https://timssandpirls.bc.edu/timss2019/methods/index.html 


For more information on the paper-based bridging study and eTIMSS see the 
forthcoming chapter: von Davier, M., Foy, P., Martin, M. O., & Mullis, |. V. S. (2020). 
Examining eTIMSS country differences between eTIMSS data and bridge data: A look at 
country-level mode of administration effects. In M. O. Martin, M. von Davier, & |. V. S. 
Mullis (Eds.), Methods and Procedures: TIMSS 2019 Technical Report (pp. 13.1-13.24). 
Retrieved from Boston College, TIMSS & PIRLS International Study Center 

website: https://timssandpirls.bc.edu/timss2019/methods/chapter-13.html 


For the TIMSS 2019 Encyclopedia see: Mullis, |. V. S., Martin, M. O., Goh, S., & Cotter, 
K. (Eds.) (2019). TIMSS 2019 Encyclopedia: Education Policy and Curriculum in 
Mathematics and Science. Available at: https://timssandpirls.bc.edu/timss2019/ 


For the TIMSS International Report 2019 see: 
https://timssandpirls.bc.edu/timss2019/ 
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Appendix C: TIMSS 2019 international benchmarks‘ 


Table 26: Summary of advanced international benchmarks of mathematics 
achievement at years 5 and 9. Pupils reaching the advanced international 
benchmark achieved a score of 625 or above 


Year 5 


Year 9 


Students can apply their understanding 
and knowledge in a variety of relatively 
complex situations and explain their 
reasoning. 


Students can solve a variety of multistep 
word problems involving whole numbers 
and show an understanding of fractions 
and decimals. They can apply 
knowledge of two- and three- 
dimensional shapes in a variety of 
situations. Students can interpret and 
represent data to solve multistep 
problems. 


Students can apply and reason in a variety 
of problem situations, solve linear equations, 
and make generalisations. 


They can solve a variety of fraction, 
proportion, and per cent problems and 
justify their conclusions. They can 
understand linear functions and algebraic 
expressions. Students can use their 
knowledge of geometric figures to solve a 
wide range of problems involving angles, 
area, and surface area. They can calculate 
means and medians, and understand how 
changing data points can impact the mean. 
Students can interpret a wide variety of data 
displays to draw and justify conclusions, and 
solve multistep problems. They can solve 
problems involving expected values. 


102 Source: TIMSS International Report 2019 
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Table 27: Summary of high international benchmarks of mathematics achievement 
at years 5 and 9. Pupils reaching the high international benchmark achieved a 
score of 550 or above 


Year 5 


Year 9 


Students apply conceptual 
understanding to solve problems. 


They can apply conceptual 
understanding of whole numbers to 
solve two-step word problems. They 
show understanding of the number line, 
multiples, factors, and rounding 
numbers, and operations with fractions 
and decimals. Students can solve 
simple measurement problems. They 
demonstrate understanding of 
geometric properties of shapes and 
angles. Students can interpret and use 
data in tables and a variety of graphs to 
solve problems. 


Students can apply their understanding and 
knowledge in a variety of relatively complex 
situations. 


They can solve problems with fractions, 
decimals, ratios, and proportions. Students 
at this level show basic procedural 
knowledge related to algebraic expressions 
and equations. They can solve a variety of 
problems with angles, including problems 
involving triangles, parallel lines, rectangles, 
and congruent and similar figures. Students 
can interpret data in a variety of graphs and 
solve simple problems involving outcomes 
and probabilities. 


Table 28: Summary of intermediate international benchmarks of mathematics 
achievement at years 5 and 9. Pupils reaching the intermediate international 
benchmark achieved a score of 475 or above 


Year 5 


Students can apply basic mathematical 
knowledge in simple situations. 


They can compute with three- and four- 
digit whole numbers in a variety of 
situations. They have some 
understanding of decimals and 
fractions. Students can identify and 
draw shapes with simple properties. 
They can read, label, and interpret 
information in graphs and tables. 


Year 9 


Students can apply basic mathematical 
knowledge in a variety of situations. 


They can solve problems involving whole 
numbers, negative numbers, fractions, 
decimals, and ratios. Students have some 
basic knowledge about properties of two- 
dimensional shapes. They can read and 
interpret data in graphs and have some 
rudimentary knowledge of probability. 
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Table 29: Summary of low international benchmarks of mathematics achievement 
at years 5 and 9. Pupils reaching the low international benchmark achieved a score 
of 400 or above 


Year 5 


Year 9 


Students have some basic 
mathematical knowledge. 


They can add, subtract, multiply, and 
divide one- and two-digit whole 
numbers. They can solve simple word 
problems. They have some knowledge 
of simple fractions and common 
geometric shapes. Students can read 
and complete simple bar graphs and 
tables. 


Students have some knowledge of whole 
numbers and basic graphs. 
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Table 30: Summary of advanced international benchmarks of science achievement 
at years 5 and 9. Pupils reaching the advanced international benchmark achieved a 
score of 625 or above 


Year 5 


Year 9 


Students communicate their 
understanding of life, physical, and Earth 
sciences and demonstrate some 
knowledge of the process of scientific 


inquiry. 


Students demonstrate knowledge of 
characteristics and life processes of a 
variety of organisms. They can 
communicate understanding of 
relationships in ecosystems and 
interactions between organisms and their 
environment. They communicate 
understanding of properties and states of 
matter and physical and chemical 
changes. Students communicate 
understanding of Earth’s physical 
characteristics, processes, and history 
and show knowledge of Earth’s revolution 
and rotation. 


Students communicate understanding of 
concepts related to biology, chemistry, 
physics, and Earth science in a variety of 
contexts. 


Students can classify animals into 
taxonomic groups. They can apply 
knowledge of cell structures and their 
functions. Students show some 
understanding of diversity, adaptation, and 
natural selection. They also recognize the 
interdependence of populations of 
organisms in an ecosystem. Students 
demonstrate knowledge of the 
composition of matter and the periodic 
table of the elements. Students use 
physical properties of matter to sort, 
classify, and compare substances and 
materials. They also recognize evidence 
that a chemical reaction has occurred. 
Students communicate understanding of 
particle spacing and motion in different 
physical states. Students apply knowledge 
of energy transfer and electrical circuits, 
can relate the properties of light and 
sound to common phenomena, and 
demonstrate understanding of forces in 
everyday contexts. Students communicate 
understanding of Earth’s structure, 
physical features, and processes. They 
demonstrate knowledge of the Earth’s 
resources and their conservation. 
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Table 31: Summary of high international benchmarks of science achievement at 
years 5 and 9. Pupils reaching the high international benchmark achieved a score 
of 550 or above 


Year 5 


Year 9 


Students communicate and apply 
knowledge of life, physical, and Earth 
sciences. 


Students communicate knowledge of 
characteristics of plants, animals, and 
their life cycles, and apply knowledge of 
ecosystems and of humans’ and 
organisms’ interactions with their 
environment. Students demonstrate 
knowledge of states and properties of 
matter and of energy transfer in 
practical contexts, and show some 
understanding of forces and motion. 
Students know various facts about the 
Earth’s physical characteristics and 
show basic understanding of the Earth- 
Moon-Sun system. 


Students apply understanding of concepts 
from biology, chemistry, physics, and Earth 
science. 


Students can apply knowledge of the 
characteristics of groups of animals, life 
processes in humans, cells and their 
functions, genetic inheritance, ecosystems, 
and nutrition. Students show some 
knowledge and understanding of the 
composition and properties of matter and 
chemical reactions. They can apply basic 
knowledge of energy transformation and 
transfer, electrical circuits, properties of 
magnets, light, sound, and forces. They can 
apply knowledge of Earth’s physical 
features, processes, cycles, and history, and 
show some understanding of Earth’s 
resources and their use. 


Table 32: Summary of intermediate international benchmarks of science 
achievement at years 5 and 9. Pupils reaching the intermediate international 
benchmark achieved a score of 475 or above 


Year 5 


Students show knowledge and 
understanding of some aspects of 
science. 


Students demonstrate some basic 
knowledge of plants and animals. They 
demonstrate knowledge about some 
properties of matter and some facts 
related to electricity, and can apply 
elementary knowledge of forces and 


Year 9 


Students show and apply some knowledge 
of biology and the physical sciences. 


Students demonstrate some knowledge of 
characteristics of animals and apply 
knowledge of ecosystems. They show some 
knowledge of the properties of matter, 
chemical changes, and a few physics 
concepts. 
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Year 5 Year 9 


motion. They show some understanding 
of Earth’s physical characteristics. 


Table 33: Summary of low international benchmarks of science achievement at 
years 5 and 9. Pupils reaching the low international benchmark achieved a score of 
400 or above 


Year 5 Year 9 
Students show limited understanding of | Students show limited understanding of 
scientific concepts and limited scientific principles and concepts and limited 
knowledge of foundational science knowledge of science facts. 
facts. 


Sample items demonstrating tasks at each international benchmark are available in the 
TIMSS international exhibits’. 


103 Source: TIMSS International Report 2019 
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Appendix D: Interpreting benchmark charts 


There are a number of charts in this report that illustrate the percentage of pupils 
reaching each of the international benchmarks of achievement. This appendix provides 
guidance on how to interpret these charts. 


Figure 149 below shows the trend over time in the percentage of year 5 pupils reaching 
each of the benchmarks in mathematics. Looking at the England 2019 row, the chart tells 
us that in 2019: 


21% of year 5 pupils in England reached the advanced benchmark 
53% of year 5 pupils in England reached the high benchmark or above 
83% reached the intermediate benchmark or above and 

96% reached the low benchmark or above 

4% of pupils (100% - 96%) did not reach the low benchmark 


The 96% of year 5 pupils in England who achieved the low benchmark in mathematics in 
2019 is a cumulative figure, which means it can be interpreted as the percentage of 
pupils who reached at least the low benchmark. This means it includes all the pupils who 
reached the low benchmark but also reached the intermediate, high and advanced 
benchmarks. 


It is possible to calculate the percentage of pupils who reached the low benchmark but no 
higher using the cumulative data displayed on the chart. Table 33 summarises the 
cumulative data shown in Figure 149. Table 34 summarises the data that can be derived 
from the chart, highlighting the calculations underpinning each cell. 
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Figure 149: (Figure 2 from Chapter 3): Trend in the percentage of year 5 pupils 
reaching each of the TIMSS international benchmarks in mathematics (England) 


International median 


2019 
England 2003 


England 1995 


0% 


= Advanced 


20% 


= High 


40% 


Percentage of pupils 


= Intermediate 


80% 100% 


Low 


Source: TIMSS 2019. 


Table 34: Trend in the percentage of year 5 pupils reaching each of the TIMSS 
international benchmarks in mathematics (England) 


Country 


International 
median 2019 


England 2019 
England 2015 
England 2011 
England 2007 
England 2003 
England 1995 


Advanced At least 


7T% 


21% 
17% 
18% 
16% 
14% 
T% 


high 
34% 71% 
53% 83% 
49% 80% 
49% 78% 
48% 79% 
43% 715% 
24% 54% 
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At least 
intermediate 


At least 
low 


92% 


96% 
96% 
93% 
94% 
93% 
82% 


Source: TIMSS 2019. 


Table 35: Trend in the percentage of year 5 pupils reaching each of the TIMSS 
international benchmarks in mathematics (England) 


Country Pupil Pupil Pupil Pupil Pupil 
reached scored scored reached scored 
advanced above the above the the low below the 
benchmark _§ intermediate low benchmark low 
benchmark benchmark but no benchmark 
and reached and reached higher 
the high the 
benchmark _ intermediate 
but no benchmark 
higher but no 
higher 

International 7% 27% 37% 21% 8% 
median (34%-7%) (71%-34%) (92%-71%) (100%- 
2019 92%) 
England 21% 32% 30% 13% 4% 
2019 
England 17% 32% 31% 16% 4% 
2015 
England 18% 31% 29% 15% 7% 
2011 
England 16% 32% 31% 15% 6% 
2007 
England 14% 29% 32% 18% T% 
2003 
England T% 17% 30% 28% 18% 
1995 


Source: TIMSS 2019. 
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